Skip to content

bench: harden and extend the parquet_row_filter_skip suite - #3

Merged
zhuqi-lucas merged 1 commit into
zhuqi-lucas:bench/parquet-row-filter-skipfrom
pydantic:claude/datafusion-pr-24328-benchmarks-tmnv3b
Aug 14, 2026
Merged

zhuqi-lucas merged 1 commit into
zhuqi-lucas:bench/parquet-row-filter-skipfrom
pydantic:claude/datafusion-pr-24328-benchmarks-tmnv3b

Conversation

@adriangb

Copy link
Copy Markdown
  • Remove the invalid top-level 'name' key from the .suite file: RawSuite is #[serde(deny_unknown_fields)] and discover_suites propagates the parse error, so benchmark_runner failed for every suite, not just this one (bench.sh drives 'cargo bench --bench sql' directly and never parses .suite files, which is why runs through bench.sh still worked). Declare the PRED_ROWS/RG_SIZE knobs as suite options while there, so benchmark_runner exposes --rows/--rg-size.

  • Make the clustering invariant explicit: add ORDER BY to the COPY so the file is guaranteed to be written in key order instead of relying on the physical planner never repartitioning the generate_series scan.

  • Strengthen the asserts: exact row count, exact matching-row count (also a correctness canary for the skip itself), and a check that pushdown_filters actually took effect so config drift cannot silently turn the suite into a no-op. Settings move from the load script to an init script, matching the clickbench suites.

  • Add coverage beyond the single best-case query: skip/q02 (predicate matches everything, all row groups fully matched), skip/q03 (filter column projected, so the skip cannot avoid decoding it), and a control subgroup on a scrambled key where no row group is ever fully matched, measuring the overhead of the check when it cannot fire.

  • Document the suite and the RG_SIZE knob in sql_benchmarks/README.md.

Claude-Session: https://claude.ai/code/session_01S1i7A2gz58sXRWiDyk9ijj

Which issue does this PR close?

  • Closes #.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

- Remove the invalid top-level 'name' key from the .suite file: RawSuite is
  #[serde(deny_unknown_fields)] and discover_suites propagates the parse
  error, so benchmark_runner failed for every suite, not just this one
  (bench.sh drives 'cargo bench --bench sql' directly and never parses
  .suite files, which is why runs through bench.sh still worked). Declare
  the PRED_ROWS/RG_SIZE knobs as suite options while there, so
  benchmark_runner exposes --rows/--rg-size.

- Make the clustering invariant explicit: add ORDER BY to the COPY so the
  file is guaranteed to be written in key order instead of relying on the
  physical planner never repartitioning the generate_series scan.

- Strengthen the asserts: exact row count, exact matching-row count (also a
  correctness canary for the skip itself), and a check that
  pushdown_filters actually took effect so config drift cannot silently
  turn the suite into a no-op. Settings move from the load script to an
  init script, matching the clickbench suites.

- Add coverage beyond the single best-case query: skip/q02 (predicate
  matches everything, all row groups fully matched), skip/q03 (filter
  column projected, so the skip cannot avoid decoding it), and a control
  subgroup on a scrambled key where no row group is ever fully matched,
  measuring the overhead of the check when it cannot fire.

- Document the suite and the RG_SIZE knob in sql_benchmarks/README.md.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01S1i7A2gz58sXRWiDyk9ijj
@adriangb

Copy link
Copy Markdown
Author

cc @zhuqi-lucas

@zhuqi-lucas
zhuqi-lucas merged commit 811dfb0 into zhuqi-lucas:bench/parquet-row-filter-skip Aug 14, 2026
16 checks passed
@zhuqi-lucas

Copy link
Copy Markdown
Owner

Thank you @adriangb , merged your suggestion!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants