You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 0bc3b6c
Browse filesBrowse the repository at this point in the historyBrowse files
- Bound the merge of spill files' output batches by the spill bound, so wide rows
cannot build batch_size-row outputs outside the memory pool.
- Bound spilled batches by a quarter of sort_spill_reservation_bytes (at least
1 MiB), so the minimum two-file merge fits in the headroom; 0 means no bound.
- Compact dictionaries, view arrays and list views at any depth, and only those;
record the size a reader decodes, including IPC padding.
- Measure used dictionary values with a bitmap instead of garbage collecting.
- Count rows per written piece; make SpillManager::with_max_batch_bytes crate-private.
- Tests: check read-back sizes against the recorded size, nested dictionaries,
list views, unsplit dictionary chunks, and that the sort spills and bounds its
merged batches.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KH2jVfxgqF5R3J7wxqvbTn
0 commit comments