Repository navigation
Commit 70f2ed0
bench: add hj queries for partitioned joins on dense and sparse integer keys (#27090)
## Which issue does this PR close?
- This PR does not close an issue. It adds a baseline for a follow-up PR
that changes how a `Partitioned` hash join with dense integer keys
filters its probe side.
## Rationale for this change
No `hj` query runs a `Partitioned` join on integer keys whose build
partitions are too large for an IN list. Such a join builds an
`ArrayMap` per partition when the keys are dense, and a hash table when
they are sparse. The suite has no query of either kind, so a change to
how these joins filter the probe side has nothing to compare against.
## What changes are included in this PR?
One commit. It adds two `hj` queries over TPC-H `orders`, joined to
itself on `o_custkey` in `Partitioned` mode:
| Query | Build side | Keys |
|---|---|---|
| Q26 | orders of the first 40K customers (~270K rows) | dense, about 10
rows per key |
| Q27 | orders of customers whose key is a multiple of 37 | sparse |
Both probe 15M rows with about a 2.7% hit rate. The runner now also
registers `orders`. The queries set no Parquet options, so a caller can
run them with or without filter pushdown, for example with
`DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true`.
## What is the testing strategy for this PR?
The change adds benchmark queries only. I ran both queries on TPC-H SF10
(16 partitions, 10 iterations) and checked their plans: both use
`HashJoinExec: mode=Partitioned`. With pushdown on, Q26's probe scan
gets a per-partition routed filter over `ArrayMap` partitions, and Q27's
gets a merged hash lookup.
| Query | `pushdown_filters=false` | `pushdown_filters=true` |
|---|---|---|
| Q26 (dense) | 50.0 ms | 93.5 ms |
| Q27 (sparse) | 54.5 ms | 59.2 ms |
## Are there any user-facing changes?
No.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <[email protected]>1 parent aeba33f commit 70f2ed0
1 file changed
Lines changed: 27 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
505 | 505 | | |
506 | 506 | | |
507 | 507 | | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
508 | 534 | | |
509 | 535 | | |
510 | 536 | | |
| |||
532 | 558 | | |
533 | 559 | | |
534 | 560 | | |
535 | | - | |
| 561 | + | |
536 | 562 | | |
537 | 563 | | |
538 | 564 | | |
| |||
0 commit comments