Skip to content

Commit c7be84a

Browse files
authored
perf: Reduce record batch memory accounting overhead (apache#24319)
## Which issue does this PR close? - Closes apache#24310. ## Rationale for this change Record batch memory accounting runs on hot execution paths. The current implementation materializes ArrayData for every array and allocates a hash set even for small batches, adding measurable overhead to queries that frequently update memory reservations. ## What changes are included in this PR? - Traverse Arrow arrays directly and recursively count their backing buffers without materializing ArrayData. - Track the first 16 buffer identities inline, then promote to a hash set for wider batches or counters spanning many batches. - Preserve shared-buffer deduplication and full buffer-capacity accounting semantics. - Add parity coverage against the previous ArrayData traversal for primitive, binary/view, list/view, fixed-size, struct, union, dictionary, map, and all legal run-end index layouts. - Add a focused Criterion benchmark across column counts, row counts, and primitive/list/struct layouts. Criterion point estimates from `cargo bench -p datafusion-common --bench record_batch_memory`, measured sequentially on upstream `main` and this PR on the same machine: ### Column count (8,192 rows, Int64) | Columns | main | this PR | Speedup | |---:|---:|---:|---:| | 1 | 81.999 ns | 10.645 ns | 7.70x | | 4 | 308.11 ns | 38.669 ns | 7.97x | | 16 | 1.2543 us | 173.55 ns | 7.23x | | 64 | 4.7457 us | 1.3914 us | 3.41x | ### Row count (4 Int64 columns) | Rows | main | this PR | Speedup | |---:|---:|---:|---:| | 1 | 299.20 ns | 39.939 ns | 7.49x | | 128 | 302.99 ns | 38.914 ns | 7.79x | | 8,192 | 307.64 ns | 39.740 ns | 7.74x | | 65,536 | 310.61 ns | 38.994 ns | 7.97x | ### Array layout (4 columns, 8,192 rows) | Layout | main | this PR | Speedup | |---|---:|---:|---:| | Primitive Int64 | 311.01 ns | 40.140 ns | 7.75x | | List of Int64 | 778.29 ns | 82.865 ns | 9.39x | | Struct of two Int64 fields | 1.0072 us | 113.70 ns | 8.86x | Each list row contains two Int64 values. Each struct column contains two Int64 child fields. ## Are these changes tested? Yes. - `cargo fmt --all -- --check` - `cargo clippy --all-targets --all-features -- -D warnings` - `cargo test -p datafusion-common utils::memory --lib` (10 passed) - `RUST_BACKTRACE=1 cargo test --profile ci --exclude datafusion-examples --exclude datafusion-benchmarks --exclude datafusion-cli --workspace --lib --tests --bins --features avro,json,backtrace,extended_tests,recursive_protection,parquet_encryption` - Focused Criterion comparison shown above ## Are there any user-facing changes? No API or behavior changes. This reduces CPU and allocation overhead in record batch memory accounting. AI assistance: OpenAI Codex assisted with implementation and test execution. I reviewed the change and its behavior end to end.
1 parent bb038a6 commit c7be84a

3 files changed

Lines changed: 577 additions & 32 deletions

File tree

‎datafusion/common/Cargo.toml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,10 @@ name = "scalar_to_array"
6464
harness = false
6565
name = "stats_merge"
6666

67+
[[bench]]
68+
harness = false
69+
name = "record_batch_memory"
70+
6771
[dependencies]
6872
arrow = { workspace = true }
6973
arrow-ipc = { workspace = true }
Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
// Licensed to the Apache Software Foundation (ASF) under one
2+
// or more contributor license agreements. See the NOTICE file
3+
// distributed with this work for additional information
4+
// regarding copyright ownership. The ASF licenses this file
5+
// to you under the Apache License, Version 2.0 (the
6+
// "License"); you may not use this file except in compliance
7+
// with the License. You may obtain a copy of the License at
8+
//
9+
// http://www.apache.org/licenses/LICENSE-2.0
10+
//
11+
// Unless required by applicable law or agreed to in writing,
12+
// software distributed under the License is distributed on an
13+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
// KIND, either express or implied. See the License for the
15+
// specific language governing permissions and limitations
16+
// under the License.
17+
18+
//! Measures the CPU overhead of accounting for the backing buffers retained by
19+
//! [`RecordBatch`]es. Batch construction is intentionally outside the timed
20+
//! region so the benchmarks isolate buffer traversal and identity deduplication.
21+
22+
use std::hint::black_box;
23+
use std::sync::Arc;
24+
25+
use arrow::array::{ArrayRef, Int64Array, ListArray, StructArray};
26+
use arrow::datatypes::{DataType, Field, Int64Type, Schema};
27+
use arrow::record_batch::RecordBatch;
28+
use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main};
29+
use datafusion_common::utils::memory::{
30+
RecordBatchMemoryCounter, get_record_batch_memory_size,
31+
};
32+
33+
fn make_batch(columns: Vec<ArrayRef>) -> RecordBatch {
34+
let fields = columns
35+
.iter()
36+
.enumerate()
37+
.map(|(index, column)| {
38+
Field::new(format!("col_{index}"), column.data_type().clone(), false)
39+
})
40+
.collect::<Vec<_>>();
41+
42+
RecordBatch::try_new(Arc::new(Schema::new(fields)), columns).unwrap()
43+
}
44+
45+
fn make_primitive_batch(num_rows: usize, num_columns: usize) -> RecordBatch {
46+
let columns = (0..num_columns)
47+
.map(|index| {
48+
Arc::new(Int64Array::from_iter_values(
49+
(0..num_rows).map(|value| value as i64 + index as i64),
50+
)) as ArrayRef
51+
})
52+
.collect::<Vec<_>>();
53+
54+
make_batch(columns)
55+
}
56+
57+
fn make_list_batch(num_rows: usize, num_columns: usize) -> RecordBatch {
58+
let columns = (0..num_columns)
59+
.map(|column| {
60+
Arc::new(ListArray::from_iter_primitive::<Int64Type, _, _>(
61+
(0..num_rows).map(|row| {
62+
let value = row as i64 + column as i64;
63+
Some(vec![Some(value), Some(value + 1)])
64+
}),
65+
)) as ArrayRef
66+
})
67+
.collect::<Vec<_>>();
68+
69+
make_batch(columns)
70+
}
71+
72+
fn make_struct_batch(num_rows: usize, num_columns: usize) -> RecordBatch {
73+
let columns = (0..num_columns)
74+
.map(|column| {
75+
let left = Arc::new(Int64Array::from_iter_values(
76+
(0..num_rows).map(|row| row as i64 + column as i64),
77+
)) as ArrayRef;
78+
let right = Arc::new(Int64Array::from_iter_values(
79+
(0..num_rows).map(|row| row as i64 - column as i64),
80+
)) as ArrayRef;
81+
82+
Arc::new(StructArray::from(vec![
83+
(Arc::new(Field::new("left", DataType::Int64, false)), left),
84+
(Arc::new(Field::new("right", DataType::Int64, false)), right),
85+
])) as ArrayRef
86+
})
87+
.collect::<Vec<_>>();
88+
89+
make_batch(columns)
90+
}
91+
92+
fn benchmark_column_count(c: &mut Criterion) {
93+
let mut group = c.benchmark_group("record_batch_memory_size/column_count");
94+
95+
// Each primitive column contributes a distinct backing buffer, exercising
96+
// both the inline buffer-ID path and hash-set promotion.
97+
for num_columns in [1, 4, 16, 64] {
98+
let batch = make_primitive_batch(8192, num_columns);
99+
group.bench_with_input(
100+
BenchmarkId::from_parameter(num_columns),
101+
&batch,
102+
|bencher, batch| {
103+
bencher.iter(|| get_record_batch_memory_size(black_box(batch)));
104+
},
105+
);
106+
}
107+
108+
group.finish();
109+
}
110+
111+
fn benchmark_row_count(c: &mut Criterion) {
112+
let mut group = c.benchmark_group("record_batch_memory_size/row_count");
113+
114+
// Buffer traversal should depend on the number of buffers, not the number
115+
// of values stored in each buffer.
116+
for num_rows in [1, 128, 8192, 65_536] {
117+
let batch = make_primitive_batch(num_rows, 4);
118+
group.bench_with_input(
119+
BenchmarkId::from_parameter(num_rows),
120+
&batch,
121+
|bencher, batch| {
122+
bencher.iter(|| get_record_batch_memory_size(black_box(batch)));
123+
},
124+
);
125+
}
126+
127+
group.finish();
128+
}
129+
130+
fn benchmark_array_layout(c: &mut Criterion) {
131+
let mut group = c.benchmark_group("record_batch_memory_size/array_layout");
132+
133+
// Compare direct primitive-buffer accounting with recursive traversal of
134+
// representative nested layouts.
135+
for (name, batch) in [
136+
("primitive", make_primitive_batch(8192, 4)),
137+
("list", make_list_batch(8192, 4)),
138+
("struct", make_struct_batch(8192, 4)),
139+
] {
140+
group.bench_with_input(
141+
BenchmarkId::from_parameter(name),
142+
&batch,
143+
|bencher, batch| {
144+
bencher.iter(|| get_record_batch_memory_size(black_box(batch)));
145+
},
146+
);
147+
}
148+
149+
group.finish();
150+
}
151+
152+
fn benchmark_shared_slices(c: &mut Criterion) {
153+
let mut group = c.benchmark_group("record_batch_memory_size/shared_slices");
154+
155+
// Model the hash-join build-side workload: one counter is reused across a
156+
// sequence of zero-copy batch slices that retain the same backing buffers.
157+
// Slicing happens outside the timed region; the benchmark measures repeated
158+
// identity lookups and the one-time accounting of each shared buffer.
159+
for num_columns in [4, 16, 64] {
160+
let batch = make_primitive_batch(8192, num_columns);
161+
let slices = (0..32)
162+
.map(|index| batch.slice(index * 256, 256))
163+
.collect::<Vec<_>>();
164+
165+
group.bench_with_input(
166+
BenchmarkId::from_parameter(num_columns),
167+
&slices,
168+
|bencher, slices| {
169+
bencher.iter(|| {
170+
let mut counter = RecordBatchMemoryCounter::new();
171+
for batch in black_box(slices) {
172+
black_box(counter.count_batch(black_box(batch)));
173+
}
174+
black_box(counter.memory_usage())
175+
});
176+
},
177+
);
178+
}
179+
180+
group.finish();
181+
}
182+
183+
criterion_group!(
184+
benches,
185+
benchmark_column_count,
186+
benchmark_row_count,
187+
benchmark_array_layout,
188+
benchmark_shared_slices
189+
);
190+
criterion_main!(benches);

0 commit comments

Comments
 (0)