Repository navigation
Commit 0b39348
Avoid formatting errors for expected missing physical columns (apache#25664)
## Which issue does this PR close?
Closes apache#25663.
## Rationale for this change
Schema adaptation can legitimately encounter columns absent from an
older physical schema. A missing-column lookup currently formats an
error listing every physical field, then discards it. This adds
avoidable latency as schemas widen.
## What changes are included in this PR?
Use the non-error-producing field lookup in the physical expression
adapter. Add a Criterion benchmark for present and missing columns in
16- and 128-field schemas.
## What is the testing strategy for this PR?
The existing adapter tests cover missing nullable and non-nullable
columns and present columns. The full extended workspace test command
passed locally, as did formatting and strict package-level Clippy.
Full-workspace Clippy currently reports two warnings in unchanged
`datafusion/physical-plan/benches/multi_group_by.rs`.
On Apple Silicon, the missing-column benchmark changed from 537 to 256
ns for 16 fields and 1.83 us to 302 ns for 128 fields. Present-column
timings were within run-to-run variation. These are microbenchmarks, not
end-to-end query measurements.
## Are there any user-facing changes?
No behavior or API changes; this reduces work for expected missing
columns.1 parent 105585e commit 0b39348
4 files changed
Lines changed: 59 additions & 1 deletion
File tree
- datafusion/physical-expr-adapter
- benches
- src
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
772 | 772 | | |
773 | 773 | | |
774 | 774 | | |
775 | | - | |
| 775 | + | |
| 776 | + | |
776 | 777 | | |
777 | 778 | | |
778 | 779 | | |
| |||
0 commit comments