Repository navigation
Commit 94c1c16
committed
fix: preserve external sort workspace across spilling
Keep acquired spill workspace available to the sorter's cursor, row, and
batch reservations instead of returning it to the execution pool before
the spill merge. Reuse idle workspace for chunked sorted output while
keeping additional output growth under the original sort consumer.
Release the idle reserve after the final output path has selected its
budget, preserving ownership and accounting for live reservations.
Add regressions for reduced memory availability, intermediate and skewed
merge retries, parent pool limits, and cleanup on completion or cancellation.
Closes apache#24739.1 parent d1fc4b3 commit 94c1c16
6 files changed
Lines changed: 1598 additions & 26 deletions
File tree
- datafusion/physical-plan/src/sorts
- sort
0 commit comments