Repository navigation
Commit 710e74e
authored
Add focused benchmark for Parquet pruning setup cache (#24323)
## Which issue does this PR close?
* Part of #21554
## Rationale for this change
`ParquetPruningSetupCache` in #21566 needs a focused benchmark that
exercises it under deliberately cache-favourable conditions.
This benchmark provides a workload where many Parquet files share the
same physical schema and use the same predicate, target partition count,
and query, creating repeated opportunities to reuse pruning setup. It is
intended to measure end-to-end scan cost for this specific workload
shape, rather than represent general DataFusion performance or predict
improvements in broader benchmarks such as ClickBench.
## What changes are included in this PR?
* Adds a Criterion benchmark that creates 128 Parquet files sharing one
physical schema and repeatedly executes `SELECT id FROM t WHERE id >=
0`.
* Keeps file generation, runtime creation, table registration, and
initial correctness validation outside the timed Criterion loop.
* Validates that the query returns all 16,384 expected rows before
benchmarking.
* Enables Criterion's `async_tokio` feature so the asynchronous query
execution can be benchmarked with the Tokio runtime.
* Registers the new `parquet_pruning_setup_cache` benchmark target.
* Documents that the benchmark is intentionally cache-favourable and
that its results should not be interpreted as general DataFusion or
ClickBench performance results.
## Are these changes tested?
The benchmark includes a correctness check that verifies the query
returns `128 × 128 = 16,384` rows before the timed benchmark begins.
No additional tests are included in this patch.
## Are there any user-facing changes?
No. This PR adds a focused benchmark and benchmark configuration only;
it does not change user-facing DataFusion behavior or public APIs.
## LLM-generated code disclosure
This PR includes LLM-generated code and comments. All LLM-generated
content has been manually reviewed.1 parent bbf739a commit 710e74e
2 files changed
Lines changed: 115 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
| 47 | + | |
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
0 commit comments