Partial aggregate state reaches the final aggregate without a memory reservation (median, array_agg) #26705

Describe the bug

For aggregates whose state holds every input value, such as median, percentile_cont and array_agg, the state moves from the partial aggregate to the final aggregate without a memory reservation. The partial aggregate emits its whole state as one batch, then its stream ends and releases its reservation. The final aggregate reserves memory only after it has merged a batch. In between, states as large as the input are held unreserved, so a plan with more than one partition uses far more memory than the pool allows.

To Reproduce

Run each query in its own datafusion-cli process (/usr/bin/time -l is macOS; use -v on Linux):

# Query 1
/usr/bin/time -l datafusion-cli -m 300m \
  -c "SET datafusion.execution.target_partitions = 1;" \
  -c "SELECT median(v) FROM generate_series(1, 30000000) AS t(v);"

# Query 2
/usr/bin/time -l datafusion-cli -m 300m \
  -c "SET datafusion.execution.target_partitions = 4;" \
  -c "SELECT median(v) FROM generate_series(1, 30000000) AS t(v);"

Expected behavior

Query 1 is above the limit for a separate reason: its value buffer reallocates into one twice its size before freeing the old one, the same shape as #25606.

Additional context

• The partial state() moves all values into a single list value: percentile_cont.rs#L469-L495.
• AggregateStream grows its reservation only after a batch has been aggregated: aggregate_stream.rs#L407-L411.
• Counting allocations against reservations (the approach of feat: log drift between MemoryPool reservations and allocated bytes in SLT #25651) at 10M rows, the final aggregate had 96 bytes reserved while 147 MB was allocated. array_agg(s) ... GROUP BY v % 10 shows the same gap.
• One direction: the final aggregate reserves an incoming state batch before merging it, since the contract in memory_pool/mod.rs makes the operator that holds batches responsible for them.
• Related: Hash aggregate with a few large groups fails after spilling: spill batches are sized by rows, not bytes #25851, Final hash aggregate fails with ResourcesExhausted when its materialized output is larger than its table #26701.

Willingness to contribute

I'm not planning to fix this myself, so anyone is welcome to pick it up. Feel free to ping me with questions about the cause or when you'd like a review.