Repository navigation
Commit e482701
bench: keep third-party and frozen baselines out of CodSpeed (#9952)
## Summary
Second of three stacked PRs on CodSpeed flakiness. The first resizes
benchmarks, the third unifies the allocator.
The three noisiest series in the whole suite are benchmarks of code
Vortex does not own or does not change:
| Benchmark | Flips on unrelated PRs (all PRs) | Max swing |
|---|---:|---:|
| `lane_kernels.rs`: `arrow_checked_add_u32` (neon leg) | 64 (97) | 67%
|
| `lane_kernels.rs`: `arrow_checked_add_u32` (avx2 leg) | 48 (65) | 22%
|
| `collect_bool.rs`: `words_gather_scalar[65536]` (avx2 leg) | 45 (62) |
15% |
| `lane_kernels.rs`: `arrow_checked_add_u32` (avx512 leg) | 30 (41) |
22% |
`arrow_checked_add_u32` runs the arrow-rs checked add over the same data
as `lanezip_checked_add_u32`. `words_gather_scalar` runs a frozen copy
of the previous scalar `collect_bool` loop. Neither can regress because
of a pull request, so a change in their number is never actionable, and
on the walltime legs they flipped on about half of all pull requests.
## Changes
Both benchmarks stay in the tree for local `cargo bench` comparisons,
but leave CodSpeed: they lose `#[cpu_features]`, which takes them off
the walltime legs, and gain `#[cfg(not(codspeed))]`, which keeps them
out of the simulation build. Imports and fixture fields that only they
use are gated the same way. The guide gets a short section on keeping
third-party and frozen baselines out of CodSpeed.
The other arrow baselines (`arrow_narrow_*` and `arrow_widen_*` in
`lane_kernels.rs`) run in simulation, have not flipped, and are left as
they are.
Checks run:
- `cargo clippy --locked --benches -p vortex-buffer -p vortex-compute --
-D warnings`
- `RUSTFLAGS="--cfg codspeed -D warnings" cargo check --locked --benches
-p vortex-buffer -p vortex-compute`, to confirm the gated build has no
unused items
- `cargo +nightly-2026-09-10 fmt` on the touched crates
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01B6ApyxEXpjiasZbQptsypn
---
_Generated by [Claude Code](https://claude.ai/code)_
Signed-off-by: Claude <[email protected]>
Co-authored-by: Claude <[email protected]>1 parent 5c7f370 commit e482701
3 files changed
Lines changed: 33 additions & 11 deletions
File tree
- docs/developer-guide
- vortex-buffer/benches
- vortex-compute/benches
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
159 | 159 | | |
160 | 160 | | |
161 | 161 | | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
162 | 171 | | |
163 | 172 | | |
164 | 173 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
21 | 23 | | |
22 | 24 | | |
23 | 25 | | |
| |||
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
| 32 | + | |
30 | 33 | | |
31 | 34 | | |
32 | 35 | | |
| |||
123 | 126 | | |
124 | 127 | | |
125 | 128 | | |
126 | | - | |
| 129 | + | |
127 | 130 | | |
128 | 131 | | |
129 | 132 | | |
| |||
180 | 183 | | |
181 | 184 | | |
182 | 185 | | |
| 186 | + | |
183 | 187 | | |
184 | 188 | | |
185 | 189 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
20 | 23 | | |
21 | 24 | | |
22 | 25 | | |
23 | 26 | | |
24 | 27 | | |
25 | 28 | | |
26 | 29 | | |
| 30 | + | |
27 | 31 | | |
28 | 32 | | |
29 | 33 | | |
30 | 34 | | |
| 35 | + | |
31 | 36 | | |
32 | 37 | | |
33 | 38 | | |
| |||
289 | 294 | | |
290 | 295 | | |
291 | 296 | | |
| 297 | + | |
292 | 298 | | |
| 299 | + | |
293 | 300 | | |
294 | 301 | | |
295 | 302 | | |
| |||
333 | 340 | | |
334 | 341 | | |
335 | 342 | | |
| 343 | + | |
336 | 344 | | |
| 345 | + | |
337 | 346 | | |
338 | 347 | | |
339 | 348 | | |
| |||
361 | 370 | | |
362 | 371 | | |
363 | 372 | | |
364 | | - | |
| 373 | + | |
365 | 374 | | |
366 | 375 | | |
367 | 376 | | |
| |||
0 commit comments