You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 708eb43
Browse filesBrowse the repository at this point in the historyBrowse files
feat: add sort pushdown benchmark and SLT tests (apache#21213)
- Related to apache#17348
- Precursor to apache#21182
Add benchmark and integration tests for sort pushdown optimization,
split out from apache#21182 per [reviewer
request](apache#21182 (comment)).
This allows comparing benchmark results before and after the
optimization lands, and the SLT diff in apache#21182 will clearly show which
test expectations changed due to the optimization.
New `sort-pushdown` benchmark subcommand with 4 queries testing sort
elimination:
| Query | Description |
|-------|-------------|
| Q1 | `ORDER BY l_orderkey ASC` (full scan) |
| Q2 | `ORDER BY l_orderkey ASC LIMIT 100` |
| Q3 | `SELECT * ORDER BY l_orderkey ASC` (wide) |
| Q4 | `SELECT * ORDER BY l_orderkey ASC LIMIT 100` (wide) |
Usage:
\`\`\`bash
./bench.sh data sort_pushdown
./bench.sh run sort_pushdown # baseline
./bench.sh run sort_pushdown_sorted # with sort elimination
\`\`\`
- **Test A**: Non-overlapping files + WITH ORDER → Sort eliminated
(single partition)
- **Test B**: Overlapping files → SortExec retained (baseline, files in
original order)
- **Test C**: LIMIT queries (ASC sort elimination + DESC reverse scan)
- **Test D**: \`target_partitions=2\` → SPM + per-partition sort
elimination
- **Test E**: Inferred ordering from Parquet metadata (no WITH ORDER) —
single and multi partition
| File | Change |
|------|--------|
| \`benchmarks/src/sort_pushdown.rs\` | New benchmark module |
| \`benchmarks/src/lib.rs\` | Register module |
| \`benchmarks/src/bin/dfbench.rs\` | Register subcommand |
| \`benchmarks/bench.sh\` | Add data/run entries |
| \`datafusion/sqllogictest/test_files/sort_pushdown.slt\` | 5 new SLT
test groups |
- [x] \`cargo clippy -p datafusion-benchmarks\` — 0 warnings
- [x] \`cargo test -p datafusion-sqllogictest -- sort_pushdown\` — all
tests pass
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
0 commit comments