Skip to content

Commit b2d08cc

Browse files
perf(scan): merge narrow splits at boundaries only some fields of a struct have (#11128)
## Summary - Tracking Issue: #11123 **Problem:** A scan splits a file wherever any column's chunk ends, and a split decodes the whole chunk of every column it overlaps. When a struct's fields chunk at unrelated rows, as list elements and dictionary-encoded columns of a wide nested file do, the union of their boundaries leaves spans of a few dozen rows. Successive splits then each decode the same chunks of the other fields. On a 180 MB list-layout file, a full scan held 3.6 GiB of native memory. **Fix:** The struct reader marks boundaries that only some of its fields have as *soft*. `SplitBy::LayoutSubSplitting` merges spans that end at a soft boundary until they reach 8,192 rows. Boundaries that every field has are never merged, and neither are boundaries from layouts above the struct, so files whose columns chunk alike split as before. **Result:** full scan of one file, with Arrow conversion as JNI does it. Live native bytes come from a counting allocator: | | Splits | Peak live | Time | Bytes requested | |---|---|---|---|---| | List layout, 180 MB, before | 139 (smallest 70 rows) | 3.57 GiB | 1.7 s | 6.2 GB | | List layout, 180 MB, after | 60 (smallest 8,192 rows) | **1.10 GiB** | **0.6 s** | 0.99 GB | | Default layout, 111 MB, before | 59 | 224 MiB | 0.2 s | | | Default layout, 111 MB, after | 48 | 224 MiB | 0.2 s | | ## Changes - **`RowSplits` gains soft marks.** Each struct field registers into its own `RowSplits`, and the struct takes their union. Any boundary that not every field has is marked soft. - The shared-boundary intersection is computed once. On the existing 256-column split-collection benchmark, the warm median falls from about 338 ms to 3 ms. - The struct's validity child is classified like any field, so a nullable struct merges the same way at any depth. - Soft marks inherited from nested structs are kept. - A single-field struct without validity registers exactly as before. - `SplitBy::Layout` collects positions only (`BoundaryMode::PositionsOnly`), so it skips the classification. - **`SplitBy::LayoutSubSplitting`** merges spans ending at a soft boundary until they reach `DEFAULT_MIN_SPLIT_ROWS` (8,192, the writer's row block), or `max_rows` if that is smaller. It never merges a span past `max_rows`. The merge runs before the existing sub-division of wide spans. --------- Signed-off-by: Christopher Wirt <[email protected]>
1 parent d455777 commit b2d08cc

4 files changed

Lines changed: 668 additions & 18 deletions

File tree

‎vortex-file/benches/split_collection.rs‎

Lines changed: 47 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
// SPDX-License-Identifier: Apache-2.0
22
// SPDX-FileCopyrightText: Copyright the Vortex contributors
33

4-
//! Benchmarks scan split collection (`SplitBy::Layout`) over written files.
4+
//! Benchmarks scan split collection over written files, with `SplitBy::Layout` and, in the
5+
//! `*_sub_splitting` variants, the default `SplitBy::LayoutSubSplitting`.
56
//!
67
//! The default write strategy produces `struct -> zoned -> chunked -> flat` per column, so
78
//! split collection walks every chunk of every column. `cold` builds a fresh reader tree per
@@ -55,7 +56,15 @@ fn main() {
5556
const ROWS_PER_CHUNK: usize = 1024;
5657

5758
/// (columns, chunks) configurations.
58-
const CONFIGS: &[(usize, usize)] = &[(16, 64)];
59+
const CONFIGS: &[(usize, usize)] = &[
60+
(16, 8),
61+
(24, 4),
62+
// Keep the larger scaling cases for local runs without lengthening CPU simulation.
63+
#[cfg(not(codspeed))]
64+
(16, 64),
65+
#[cfg(not(codspeed))]
66+
(256, 64),
67+
];
5968

6069
static RUNTIME: LazyLock<tokio::runtime::Runtime> = LazyLock::new(|| {
6170
tokio::runtime::Builder::new_current_thread()
@@ -194,8 +203,12 @@ static MISALIGNED_FILES: LazyLock<HashMap<(usize, usize), VortexFile>> = LazyLoc
194203
});
195204

196205
fn collect_splits(file: &VortexFile) -> Vec<u64> {
206+
collect_splits_by(file, SplitBy::Layout)
207+
}
208+
209+
fn collect_splits_by(file: &VortexFile, split_by: SplitBy) -> Vec<u64> {
197210
let reader = file.layout_reader().unwrap();
198-
SplitBy::Layout
211+
split_by
199212
.splits(reader.as_ref(), &(0..file.row_count()), &[FieldMask::All])
200213
.unwrap()
201214
}
@@ -248,3 +261,34 @@ fn cold_misaligned(bencher: Bencher, config: &(usize, usize)) {
248261

249262
bencher.bench(|| collect_splits(file));
250263
}
264+
265+
/// Like `warm`, but with the default `SplitBy::LayoutSubSplitting`, which also classifies soft
266+
/// boundaries.
267+
#[divan::bench(args = CONFIGS)]
268+
fn warm_sub_splitting(bencher: Bencher, config: &(usize, usize)) {
269+
let file = &FILES[config];
270+
let reader = file.layout_reader().unwrap();
271+
let row_count = file.row_count();
272+
273+
bencher.bench(|| {
274+
SplitBy::default()
275+
.splits(reader.as_ref(), &(0..row_count), &[FieldMask::All])
276+
.unwrap()
277+
});
278+
}
279+
280+
/// Like `cold_misaligned`, but with the default `SplitBy::LayoutSubSplitting`.
281+
#[divan::bench(args = MISALIGNED_CONFIGS)]
282+
fn cold_misaligned_sub_splitting(bencher: Bencher, config: &(usize, usize)) {
283+
let file = &MISALIGNED_FILES[config];
284+
// Sanity-check once per config that the file has soft boundaries to merge, so this measures
285+
// the classification path rather than the cheap one.
286+
let merged = collect_splits_by(file, SplitBy::default()).len();
287+
let layout = collect_splits(file).len();
288+
assert!(
289+
merged < layout,
290+
"expected soft boundaries to merge, got {merged} splits vs {layout} by layout"
291+
);
292+
293+
bencher.bench(|| collect_splits_by(file, SplitBy::default()));
294+
}

0 commit comments

Comments
 (0)