Free and account for spill range metadata in the native shuffle writer #6009

What is the problem the feature request solves?

Since #5916 the native shuffle writer spills every output partition of a task into one file and records where each partition's blocks are in PartitionedSpill.ranges, a Vec<Vec<Range<u64>>> with one 16-byte range per partition per spill round. The lists live until the task ends: finish_partition reads a partition's ranges but never drops them, and no memory reservation covers them.

That is 16 bytes per spilled block. With 16,000 output partitions and 11 spill rounds it is about 2.8 MB per task; a pool under pressure that spills every batch can reach 16,000 partitions × 88 rounds ≈ 22 MB per task, about 180 MB for an executor running 8 tasks, none of it visible to the memory pool.

Describe the potential solution

• Drop a partition's range list once finish_partition has copied it into the output.
• Grow the repartitioner's MemoryReservation by the range size when a range is pushed and shrink it at merge. Use the infallible grow, so a spill cannot fail because of its own bookkeeping.
• Measure retained range memory under heavy spilling (many partitions, many rounds) before and after.

Additional context

Suggested in the review of #5916: #5916 (comment)