But you can use summary fields in calculations. Do it all the time.
If you use a summary field in a calculation, it's evaluation context may differ from that of a summary field put on the layout if it is located inside a sub summary layout part, but that does not mean that you cannot use a summary field in a calculation. Summary fields used inside a calculation return a "grand total" result unless you use the GetSummary function to access subtotals produced by sorting your records much like you can by putting summary fields inside a sub summary layout part.
I suspect, however, that the issue here is in that phrase "weighted average", can you explain more fully why you can't use the summary fields in a calculation?