You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 801b017
Browse filesBrowse the repository at this point in the historyBrowse files
perf: normalize float hash join build keys once (apache#25972)
## Which issue does this PR close?
- Follow-up to apache#25658, from @jayzhan211's review there. No separate
issue.
This branch is stacked on apache#25658, so until that merges the diff here
includes its commits. Only the last commit is new.
## Rationale for this change
Hash joins on float keys get much slower as the build side grows. The
comparator that checks key equality is built once per probe batch, and
building it rewrites `-0.0` to `+0.0` across the whole build key column
so that `-0.0 = 0.0` holds. That scan repeats for every probe batch even
though the build side never changes, so a float-key join costs build
rows times probe batches, while the same join on integer keys does not.
## What changes are included in this PR?
- `collect_left_input` normalizes the build key columns once and keeps
the result as the join's build keys. The in-list and bounds dynamic
filters and the build-side null check still read the original keys,
which they already did before this point.
- A rewritten copy is only made when a column actually holds `-0.0`;
otherwise the stored key shares the original buffer. Any copy is charged
to the build-side memory reservation.
- `JoinKeyComparator::new_with_normalized_left` and
`equal_rows_arr_with_normalized_left` skip the build-side pass. Hash
join's lookup uses them; sort-merge, piecewise merge, ASOF and symmetric
hash joins keep the existing behavior.
The query from the review, TPC-H SF1, `target_partitions = 1`,
`HashJoinExec` `join_time`, median of five interleaved runs:
```sql
SELECT count(*)
FROM (SELECT l_orderkey AS k, CAST(l_quantity AS DOUBLE) AS q FROM lineitem WHERE l_orderkey < 1500000) b
JOIN (SELECT l_orderkey AS k, CAST(l_quantity AS DOUBLE) AS q FROM lineitem) p
ON b.k = p.k AND b.q = p.q;
```
| keys | before | after |
|---|---|---|
| `(BIGINT, DOUBLE)` | 96.4 ms | 25.7 ms |
| `(BIGINT, BIGINT)` control | 24.9 ms | 24.8 ms |
## What is the testing strategy for this PR?
New tests:
- `join_key_comparator_with_normalized_left_skips_left_normalization` in
`joins/utils.rs` pins the new constructor's contract.
- `collect_left_input_stores_normalized_float_keys` checks that Float64
and Float32 build keys are stored with `+0.0` while the batch keeps
`-0.0`, that no copy is made without `-0.0`, and that the copy is
charged to the reservation.
- `join_inner_single_float_key_negative_zero` covers a single float key
with `-0.0` and `+0.0` on both sides, over batch sizes and both null
equality modes.
- `test_null_aware_left_anti_float_scope_negative_zero` covers both
null-aware scope lookups with float scope keys.
`join_inner_multi_key_float_zero_and_nulls_across_chunks` and the
existing join tests and `joins`, `null_aware` and `subquery`
sqllogictest files pass unchanged.
## Are there any user-facing changes?
No.
0 commit comments