Skip to content

Commit a17bca9

Browse files
committed
Review the row framework: fix the docs, close the test gaps
A review pass over the row scalar-function framework in three parts. **The author-facing API is unchanged.** Every proposal that would have altered it was implemented, run against the suite, and then backed out, because none of them prevents a bug and this branch's open work is settling the API rather than churning it. What lands is cleanup, corrected documentation, and test coverage. The emitted IR of every `visit_prepared_into` monomorph is identical to the parent commit. Corrections, each of which was a claim in a doc comment that is not true: - The dense-retry path does not filter unconditionally because a deferred sink cannot skip rows. `ERRORS_ARE_DEFERRED` and `SUPPORTS_SKIPPED_ROWS` are independent consts and a sink may legally set both; the real reason is that `execute_dense` is never handed the `branch` closure. - `CheckedSink`'s `SUPPORTS_SKIPPED_ROWS = false` is load-bearing, not a restatement of the policy. A deferred result reaches valid-only execution whenever its arguments are not dense-safe, so the sink's own `false` is what keeps its uninitialized slots sound. - `scalar_fn`'s trait-choice doc listed `vortex.byte_length` as a `RowFn` when it is a `ScalarFnVTable`. It is now the worked example of row-determined being necessary but not sufficient: it measured 7.6x slower as a row function, because the length is a field of the view and the row loop paid to resolve bytes it never read. - `contains_route`'s bump guidance pointed at `constant_operands_agree_with_columns` as the check that catches a moved row. That test covers four of the ten container types. `routes_agree_with_geo_for_every_container` now covers all ten against plain `a.contains(b)` in every constant arrangement. - The workspace `geo` requirement admits every 0.31.x, which `cargo update` takes with no diff to review, so the lockfile is the real boundary rather than the version line. Cleanup: - Delete `InputElement::decode_null_tolerant` overrides that only restated the default, from the primitive, bool and `TensorRow` elements. `GeometryRow`'s is the only real one, and the doc now says a dense-safe element should not override. - Keep one `l2_norm_row`. Cosine's prepared and per-row arms must agree bit for bit, which holds only while both accumulate in the same order, so the second copy was an invitation to break the property its comments defend. - Zip the cosine `reduce_encoded` loops instead of indexing three slices per row, and record why it materializes where `InnerProduct` stays lazy. - Collapse the checked-arithmetic macros so each predicate is written once. `impl_checked_integer!` owns the shape; the unsigned and signed wrappers own their add/sub/div; only `mul_failure` varies per width. Delete `CheckedArithmetic::mul_error`, which had no caller once `CheckedMul` reported through `mul_failure`. - Share `resolve_validity` between `execute_filtered` and the forced-strategy test seam so the mask shortcuts cannot drift apart. - Record on `ElementTuple` why it carries arities past the widest function in tree: it is sealed, so those are the only ones a downstream crate can have. Coverage: - `reduce_encoded_is_probed_before_and_after_filtering` pins that the rewrite is offered the original arrays at full length and then the filtered ones at the surviving count. Both probes matter, which is why the #9130 follow-up to skip the first one should be struck rather than implemented: it is the only probe that sees the arrays still encoded. `SCALAR_FN_HANDOFF.md` records the five changes proposed and reverted, so they are decided rather than re-attempted: three API changes now posed as questions for #9129, the `ElementTuple` arity trim, and making `CheckedSink` safe with `BufferMut::zeroed`. That last one measured 1.65 to 1.71x slower than the `MaybeUninit` storage it replaced, across every batch size from 8 KiB to 2 MiB, because `alloc_zeroed` does not avoid the write pass the row loop then duplicates. The notes also state the IR gate's blind spot, which is what let that regression pass it: the IR of a row loop cannot show an allocator call outside the loop. Verification: 3319 vortex-array, 179 vortex-tensor and 241 vortex-geo tests, 72 doctests, clippy over the three crates with `--all-targets --all-features`, nightly fmt, and `git diff --check`. Signed-off-by: "Connor Tsui" <[email protected]> Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_014mwiAahcxc5xBfTrhDK11L
1 parent f430847 commit a17bca9

16 files changed

Lines changed: 545 additions & 282 deletions

File tree

‎Cargo.toml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -160,6 +160,10 @@ flatbuffers = "25.2.10"
160160
fsst-rs = "0.6.0"
161161
futures = { version = "0.3.31", default-features = false }
162162
fuzzy-matcher = "0.3"
163+
# `vortex-geo`'s `contains_route` transcribes geo's `impl_contains_from_relate!` dispatch table, so
164+
# any bump that moves a row silently changes containment verdicts. This requirement admits 0.31.x,
165+
# which `cargo update` takes without a diff here, so the lockfile is as load-bearing as this line;
166+
# see `vortex-geo/src/scalar_fn/contains.rs`.
163167
geo = "0.31.0"
164168
geo-traits = "0.3.0"
165169
geo-types = "0.7.19"

‎SCALAR_FN_HANDOFF.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -278,6 +278,140 @@ The checks recorded for the final API state are:
278278
The generated-code comparison and native timing evidence are described above and in the final
279279
section of `STRICT_SCALAR_FN_RESEARCH.md`.
280280

281+
## Review pass: what changed and what was deliberately left
282+
283+
A review of the three parts (API, execution, implementations). **The author-facing API is
284+
unchanged**: every proposal that would have altered it was backed out, for the reasons below, and
285+
what landed is cleanup, corrected documentation, and test coverage. The emitted IR of every
286+
`visit_prepared_into` monomorph is identical to the pre-review commit.
287+
288+
API:
289+
290+
- `InputElement::decode_null_tolerant` overrides that only restated the default were deleted from
291+
the primitive, bool and `TensorRow` elements. `GeometryRow`'s override is the only real one. The
292+
doc now says a dense-safe element should *not* override.
293+
- `ElementTuple` now records why it carries arities past the widest function in tree: it is sealed,
294+
so a downstream crate cannot add the one it needs, and an uninstantiated arity costs only its own
295+
macro expansion.
296+
297+
Execution:
298+
299+
- `execute_filtered` and the forced-strategy test seam now share `resolve_validity`, so the mask
300+
materialization and the all-true/all-false shortcuts cannot drift apart between them.
301+
- The dense-retry path's comment was wrong and is corrected. It filters unconditionally because
302+
`execute_dense` is not handed the `branch` closure, **not** because a deferred sink cannot skip
303+
rows: `ERRORS_ARE_DEFERRED` and `SUPPORTS_SKIPPED_ROWS` are independent consts and a sink may
304+
legally set both.
305+
306+
Implementations:
307+
308+
- `l2_norm_row` had two copies, in `l2_norm.rs` and `cosine_similarity.rs`. Cosine's prepared and
309+
per-row arms must agree bit for bit, which only holds while both accumulate in the same order, so
310+
the duplicate was an invitation to break exactly the property the comments defend. One copy now
311+
lives in `scalar_fns/row.rs`.
312+
- `CosineSimilarity::reduce_encoded` zips its three slices instead of indexing `0..len` three times
313+
per row, and documents why it materializes where `InnerProduct::reduce_encoded` stays lazy (the
314+
zero-norm guard is a conditional, not an arithmetic factor).
315+
- The checked-arithmetic macros collapsed from four near-identical bodies into one
316+
`impl_checked_integer!` parameterized by the error tests and the `mul_failure` form, which is the
317+
only thing that genuinely varies. `CheckedArithmetic::mul_error` was deleted: `CheckedMul` reports
318+
through `mul_failure`, and after the collapse nothing called the boolean form.
319+
- `contains_route` and the workspace `geo` dependency both record that the table transcribes geo's
320+
`impl_contains_from_relate!` and must be re-verified on a version bump. `geo = "0.31.0"` admits
321+
patches but not `0.32`, so the coupling can only break deliberately.
322+
323+
Deliberately **not** done:
324+
325+
- **No `DeferredElementSink`.** `CheckedSink` exists largely because `ElementSink` cannot name an
326+
error at `finish`. A framework sink combining an element output with a type-level message would
327+
remove ~100 lines per function, but there is exactly one deferred-error function. Build it when a
328+
second appears, rather than copying `CheckedSink`.
329+
- **No change to `reduce_encoded`'s probe semantics.** Hoisting the probe out of the strategy paths
330+
and masking a full-length result looks like a simplification and is not one:
331+
`normalized_readthrough_survives_null_rows` pins that a filtered input is no longer `Normalized`,
332+
so which arrays reach `reduce_encoded` is load-bearing and differs per strategy.
333+
- **No PR split.** Recommended landing order, each step individually revertible and separately
334+
benchmarkable: (1) API + lifting with dense/filter only; (2) branch-and-skip + adaptive selection
335+
+ its benchmarks; (3) `NumericBinary`; (4) tensor; (5) geo. The seam already supports this split
336+
and no API changes between steps.
337+
338+
### Three API changes proposed, and why none of them landed
339+
340+
All three were implemented, run against the suite, and backed out. None prevents a bug, and this
341+
branch's open work is *settling* the API rather than churning it, so they belong in #9129 as
342+
questions decided alongside the rest of the surface:
343+
344+
- **Should `reduce_encoded` take an explicit `row_count`?** The filtered-count requirement is real
345+
and easy to miss, but `args` are filtered to match, so `args[0].len()` is already both the natural
346+
thing to write and correct. The parameter is documentation, and it costs every implementor a
347+
signature change. What survived is the test:
348+
`reduce_encoded_is_probed_before_and_after_filtering` pins that the rewrite is offered the
349+
original arrays at full length and then the filtered ones at the surviving count.
350+
- **Should `OutputSink::row_count_matches` become `rows_len`?** A length reads cleaner and lets the
351+
executor name what it found. Against that, `row_count_matches` lets a sink fold in its own
352+
invariants, which `SpreadSink` uses for its width check; narrowing it turns that into a panic.
353+
Neither spelling prevents a bug.
354+
- **Should the nullary path go?** A function with no inputs has no validity to lift, which is the
355+
lifting's whole job. But `RowFn` would still give it sink allocation and dtype derivation, so
356+
`random()` or `now()` is not obviously better hand-written, and the path is ~70 lines and tested.
357+
358+
Trimming `ElementTuple` to arity four was proposed on the same reasoning and backed out for a
359+
stronger one: the trait is sealed, so the arities are the only ones a downstream crate can ever
360+
have.
361+
362+
### Two changes this pass made and then reverted
363+
364+
Both were proposed, implemented, reviewed, and backed out on evidence. They are recorded because
365+
each is an attractive idea that a later reader will have again.
366+
367+
**Making `CheckedSink` safe with `BufferMut::zeroed` costs 1.65 to 1.71x.** Replacing the
368+
`MaybeUninit` storage removes an `unsafe set_len` and reads as a clear win, and `ElementSink`'s own
369+
comment appears to bless it by routing a zeroable placeholder to `alloc_zeroed`. Measured, it is
370+
not: allocate-zeroed-then-fill against allocate-then-fill, interleaved in one process over `u64`
371+
outputs, ran **1.221x** slower at 8 KiB, **1.71x** at 64 KiB, **1.66x** at 512 KiB and **1.71x** at
372+
2 MiB, stable to within 2% across two runs. `alloc_zeroed` does not avoid the write: below glibc's
373+
mmap threshold `calloc` recycles a dirty chunk and memsets it, and above it every fresh page faults
374+
on first touch. The row loop overwrites every slot regardless, so this is a duplicated pass over
375+
the output of the hottest kernel in the system.
376+
377+
Note the corollary, which is a real optimization nobody has taken: `ElementSink::with_capacity`
378+
pays exactly this on every batch, and only branch-and-skip ever reads a placeholder back. A sink
379+
that allocated uninitialized on the dense and filter paths would recover it.
380+
381+
**Hoisting `OutputSink::SUPPORTS_SKIPPED_ROWS` into the plan is not sound as an optimization.**
382+
#9130 records "avoid probing `reduce_encoded` twice when branch execution is unsupported" as a
383+
follow-up. It reads as free, and is not, because the branch path probes `reduce_encoded` against
384+
the _original_ arrays before it consults the sink, and that is the only probe that ever sees them
385+
still encoded. Skipping the path early leaves such a function with only the filtered probe, whose
386+
canonical arrays match no encoding fast path. For a function whose reduction is _defined_ to answer
387+
differently from its row loop, which is exactly what `L2Norm` over `Normalized` is, that is a wrong
388+
answer rather than a slow one. Nothing in tree is reachable today only because every `ValidOnly`
389+
dispatch happens to use `ElementSink`. **#9130's follow-up should be struck, not implemented.**
390+
`reduce_encoded_row_count_follows_the_arrays_it_is_given` now pins the two probes and their row
391+
counts.
392+
393+
### On measurement, and what the IR gate does and does not cover
394+
395+
Wall-clock benchmarking of the row loops was attempted first and abandoned on evidence. Two runs of
396+
the *same* baseline binary, pinned with `taskset -c 2`, 100 samples, disagreed by up to 4x
397+
(`row_wrapping_add_nullable`: 198.8 us then 52.9 us median; `specialized_checked_add`: 185.5 us then
398+
34.4 us). The 4-vCPU shared VM drifts more within a session than any effect being measured, which is
399+
the same conclusion this branch already reached on a dedicated 7950X.
400+
401+
The gate used instead is the emitted optimized IR of every `visit_prepared_into` monomorph in
402+
`vortex-array`, profiled by vector width, reduction count, overflow-intrinsic survival and bounds
403+
checks, then compared as a multiset before and after. Reproduce with:
404+
405+
```bash
406+
RUSTFLAGS="--emit=llvm-ir -C codegen-units=1" cargo rustc -p vortex-array --release --lib
407+
```
408+
409+
**Its blind spot is worth stating, because it nearly landed a regression.** The IR of a row loop
410+
cannot show an allocator call outside it, so the `BufferMut::zeroed` substitution above passed this
411+
gate cleanly while costing 1.7x. An allocation-strategy change needs its own targeted A/B, which is
412+
cheap to write and immune to the host drift above because both arms run interleaved in one process.
413+
Use the IR gate for loop shape and a focused microbenchmark for anything the loop does not contain.
414+
281415
## Remaining boundaries
282416

283417
- Complete the required x86 production and forced-null-strategy benchmark run above before treating

0 commit comments

Comments
 (0)