Repository navigation
Conversation
Merging this PR will degrade performance by 8.3%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | take_fsl_f16_force_per_index[256, 10] |
38 µs | 55.5 µs | -31.54% |
| ❌ | WallTime | filtered_sink_i64_avx2[OneNullInEight] |
22.1 µs | 26.1 µs | -15.21% |
| ❌ | WallTime | decode_avx512[8192, (Inline, OneNullInEight)] |
80.5 µs | 93.5 µs | -13.87% |
| ❌ | Simulation | set_indices_vortex_buffer[128] |
1.8 µs | 2.1 µs | -10.73% |
| ⚡ | WallTime | filtered_sink_i64_avx512[OneNullInEight] |
26.3 µs | 22.3 µs | +17.94% |
| ⚡ | WallTime | scalar_subtract_neon |
13.4 µs | 11.9 µs | +12.98% |
| Simulation | fixed_16_advancing_ptr_safe[100] |
< 1 ns | < 1 ns | N/A | |
| Simulation | preverify_advancing_ptr_unchecked[1000] |
< 1 ns | < 1 ns | N/A | |
| Simulation | preverify_advancing_ptr_unchecked[10000] |
< 1 ns | < 1 ns | N/A | |
| Simulation | bench_compare_sliced_dict_primitive[(3333, 10000)] |
< 1 ns | < 1 ns | N/A |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing mk/decimal-modes (9e757a2) with mk/scheme-registry (80af2c5)2
Footnotes
-
389 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
mk/scheme-registry(70b0bfe) during the generation of this report, so 62c3359 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
2fb17f9 to
70b0ab1
Compare
b6464eb to
05b6a70
Compare
70b0ab1 to
830c260
Compare
05b6a70 to
5b0b07b
Compare
830c260 to
80af2c5
Compare
5b0b07b to
ddca038
Compare
80af2c5 to
70b0bfe
Compare
ddca038 to
9e757a2
Compare
Signed-off-by: Matt Katz <[email protected]>
70b0bfe to
b9329d9
Compare
Signed-off-by: Matt Katz <[email protected]>
9e757a2 to
17af133
Compare
b9329d9 to
bca8e15
Compare
…zed IDs permit it (#10076) Stacked on #10069. ## Summary Wide `i128` and `i256` decimals compress into multi-part `DecimalByteParts` arrays when the compressor's allowed serialized IDs include `vortex.decimal_byte_parts.v2`. The session registers the v1 decimal scheme, and `build()` refines it to v2 wherever both decimal IDs are allowed. This PR sets the stage for versioned schemes. ## `Scheme::refine` New defaulted hook in `vortex-compressor`: ```rust fn refine(&self, _allowed: &dyn Fn(&ArrayId) -> bool) -> Option<&'static dyn Scheme> { None } ``` - It returns the variant of the scheme to use for the allowed IDs; `None` keeps the scheme. - A variant must keep the scheme's `SchemeId`, and every ID it declares in `produced_encodings` must still be allowed for it to be used. - The allowed IDs are a predicate rather than a set, so builders that allow every ID (`empty()`, `unrestricted()`) are represented exactly. ## Builder `build()` now: 1. drops the mode's excluded schemes, 2. refines each remaining scheme against the allowed IDs, 3. keeps the schemes whose produced IDs are all allowed. The allowed IDs are the session's `AllowedIds` minus the mode's excluded serialized IDs. The `Cuda` mode excludes `vortex.decimal_byte_parts.v2`, since multi-part arrays have no CUDA decode kernel, so the CUDA preset keeps v1. ## `DecimalScheme` - `DecimalScheme` gains v1 and v2 modes with `v1()` and `v2()` constructors; `Default` is v1. The session registers v1. - v1 leaves values wider than `i64` canonical after narrowing. v2 splits them with `split_decimal` and compresses the most significant part and each lower part as children. Single-part arrays serialize as v1 in either mode. - `refine` returns v2 when both decimal IDs are allowed and v1 otherwise, from either mode. Unlike #10035's `try_upgrade`, a v2 scheme is refined down to v1 rather than dropped. ## Behaviour | Builder | Wide decimals | |---|---| | `from_session` with the core edition | canonical (v1), unchanged | | `from_session(..).disable_editions()`, including the writer with `disable_editions()` | split (v2) | | `unrestricted()` / `empty()` with the v2 scheme | split (v2) | | `only_cuda_compatible()` | canonical (v1) | Default writes are unchanged: the core edition allows v1 only. Strategies built from `WriteStrategyBuilder::from_session` keep the enabled editions' restrictions even when a write disables editions. Compared with #10035, `vortex-file`'s writer and `vortex-cuda` need no changes: the writer's `disable_editions` path already lifts the edition restriction, and the CUDA restriction lives in the `Cuda` mode. ## Tests - `vortex-btrblocks/src/schemes/decimal.rs`: `refine` for every combination of allowed decimal IDs, from both modes. - `vortex-btrblocks/tests/decimal_config.rs`: decimal output for no editions, the core edition, editions disabled, unrestricted, and the CUDA preset; excluding the decimal scheme is not undone by refinement; signed/nullable `i128`/`i256` round trips with and without child compression. - `vortex-file`: the default writer splits wide decimals only with editions disabled; strategies built from the session keep the enabled editions' restrictions. ## Checks - `cargo clippy -D warnings` on `vortex-btrblocks`, `vortex-compressor`, `vortex-file` with `--all-targets --all-features`, `--all-targets`, and `--no-default-features`: passed - `cargo nextest run --all-features -p vortex-btrblocks -p vortex-compressor -p vortex-file -p vortex -p vortex-layout -p string-bench -p vortex-compat -p vortex-decimal-byte-parts`: 988 passed, 1 skipped - `cargo test --doc -p vortex-btrblocks -p vortex-compressor`: passed - `RUSTDOCFLAGS="-D warnings" cargo doc -p vortex-btrblocks -p vortex-compressor --no-deps --all-features`: passed - `rustfmt +nightly-2026-09-10 --check` on touched files: passed - Not run: `vortex-cuda` (needs `nvcc`), benchmarks, workspace-wide clippy and tests. --------- Signed-off-by: Matt Katz <[email protected]>
Stacked on #10034.
Summary
Wide
i128andi256decimals compress into multi-partDecimalBytePartsarrays when the writer permits thevortex.decimal_byte_parts.v2wire format. The registry holds the v1 scheme, which only ever writes the frozen v1 format; permission filtering swaps in the v2 variant wherever the permitted serialized IDs allow it.Changes
vortex-compressor:Scheme::try_upgrade(&self, &HashSet<ArrayId>) -> Option<&'static dyn Scheme>is a new defaulted hook returning a newer variant the permitted IDs support; it must keep theSchemeIdand never downgrade.permit_schemes(schemes, &allowed)(the function behindCompressionSessionExt::permit) calls it for each scheme before filtering by declared IDs, and is exported for callers with an explicit permission set.vortex-btrblocks:DEFAULT_SCHEMESregistersDecimalScheme::v1().DecimalSchemegains v1 and v2 modes withv1()andv2()constructors;Defaultis v1. v1 leaves values wider thani64canonical after narrowing. v2 splits them withsplit_decimaland compresses the most significant part and each lower part as children. Single-part arrays serialize as v1 in either mode.try_upgradereturns a static v2 instance when both decimal IDs are permitted; an explicitly listed v2 is dropped, not downgraded, when either ID is missing.vortex-file: with editions disabled the writer now permits every registered serialized ID throughpermit_schemes, so the upgrade applies there too.vortex-cuda:cuda_compatible_schemespins the decimal scheme to v1 after permissions are applied, since multi-part arrays have no CUDA decode kernel.Behavior
WriteStrategyBuilder::from_sessionkeep the enabled editions' permissions even when a write disables editions.Tests
vortex-btrblocks/tests/decimal_config.rs: mode selection against the core edition and explicit ID sets, v2 on top of the core edition, explicit modes only upgrade, exclusion is not undone by an upgrade, and signed/nullablei128/i256round trips with and without child compression.vortex-file: the default writer splits wide decimals only with editions disabled; strategies built from the session keep the enabled editions' permissions.Validation
Run locally:
cargo +nightly-2026-09-10 fmton the touched crates;cargo clippy --all-targets --all-features -- -D warningsonvortex-compressor,vortex-btrblocks,vortex-file,vortex(clean);cargo nextest run --all-featuresonvortex-compressor,vortex-btrblocks,vortex-file(296 passed, 1 skipped); doctests on the same three crates (passed). Not run:vortex-cuda, which needsnvcc.