Repository navigation
Commit 83022b1
authored
SQL benchmark configuration parity updates - apache#24778 2 (apache#24803)
## Which issue does this PR close?
Part of apache#21706. This PR does not close the issue.
## Rationale for this change
The SQL-based TPC-H, TPC-DS, and sort TPC-H benchmarks did not
consistently expose the same schemas, constraints, and loading behavior
as their native Rust equivalents. These differences could affect query
planning and make benchmark results less directly comparable.
## What changes are included in this PR?
- Align the TPC-H CSV, Parquet, and in-memory table definitions with the
native benchmark's data types, nullability, and primary-key constraints.
- Add explicit TPC-DS schemas and primary-key constraints while
retaining the physical types used by the DataFusion benchmark Parquet
dataset.
- Update the sort TPC-H loader to:
- Read the complete `lineitem` Parquet directory rather than only
`lineitem.1.parquet`.
- Keep the table Parquet-backed instead of materializing it.
- Declare existing `l_orderkey ASC NULLS LAST` ordering as metadata
instead of physically sorting the data.
The sort TPC-H changes match the default behavior of the native Rust
benchmark when it is run without `--mem-table`.
## Are these changes tested?
Yes.
The following checks were run:
```shell
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
Sort TPC-H query 1 was also executed against the SF1 dataset in both
modes:
cargo run --profile release -p datafusion-benchmarks \
--bin benchmark_runner -- sort_tpch \
--scale-factor 1 --sorted false --query 1 --iterations 1
cargo run --profile release -p datafusion-benchmarks \
--bin benchmark_runner -- sort_tpch \
--scale-factor 1 --sorted true --query 1 --iterations 1
Both modes completed successfully and returned all 6,001,215 lineitem
rows.
## Are there any user-facing changes?
No public API behavior changes. The SQL benchmark definitions now more
closely match the corresponding native benchmark configurations.1 parent 07483c1 commit 83022b1
5 files changed
Lines changed: 775 additions & 105 deletions
File tree
- benchmarks/sql_benchmarks
- sort_tpch/init
- tpcds/init
- tpch/init
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
3 | | - | |
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
0 commit comments