Repository navigation
Commit 0fea4a3
bench: stop gating on benchmarks CodSpeed cannot measure (#10063)
## Summary
Four groups of benchmarks that cannot tell a reviewer anything about a
pull request. Every value
below moved on pull requests that changed no Vortex code.
| Benchmark | Values seen | Unrelated PRs |
| --- | --- | ---: |
| `allocation::allocate_drop_arrow[65536]` | 90.7 / 101.8 µs | 4 |
| `allocation::from_vec_drop_arrow[16384]` | 40.4 / 31.4 µs | 4 |
| `take_fsl::..._force_manual_range_copy[2048, 10]` | 10.2 / 63.3 µs | 7
|
| `take_fsl::..._force_per_index[256, 10]` | 40 / 56.9 µs | 4 |
| `take_chunked::take[duplicates/repeated/...]` | 24 / 104 µs | 3 |
| `take_slices_to_buffer_matrix` (3 cases) | no execution time | 26 of
26 |
| `row_fn_utf8::decode` (both legs) | 80.7 / 187.6 µs | 7 |
| `row_fn_output::filtered_*` (all legs) | 10 to 19% swings | 5 |
## Changes
- **allocation**: gate the `bytes` and Arrow cases out of CodSpeed. They
measure allocators Vortex
does not own, so no pull request here can move them. They stay for local
`cargo bench`.
- **take_fsl**: remove the three forced-strategy benchmarks. They
compare internal strategies
against each other, which is a local question, and at 10 indices the
measured region is
dominated by allocation. Two of the three flip, and both also report no
execution time on some
runs.
- **take_chunked**: drop the `repeated` duplicates case, which took one
index 1,000 times.
`duplicate90` and `duplicate99` cover heavy duplication without the
degenerate input.
- **row_fn_utf8**: remove the file. Its one benchmark swung 10 to 18%
per report on 13 of the
last 24, including on a lockfile refresh and a one-file Python fix. Both
legs move, so this is
the benchmark rather than one runner.
- **row_fn_output**: remove `filtered_owned_i64` and `filtered_sink_i64`
and their support types.
They swung 10 to 19% on avx2, avx512 and neon alike, on unrelated pull
requests. The infallible
and deferred benchmarks in the same file have stayed put and are
untouched.
- **take_slices_to_buffer_matrix**: read one output element through an
opaque index. Black-boxing
the returned buffer alone leaves the copy loop dead, because nothing
reads what it wrote into
spare capacity, which is why three cases have measured nothing on every
recent report.
### Expect a one-time rebaseline
Two kinds. The advancing-pointer numbers rise once their stores survive.
The other kind is worth understanding: removing a benchmark speeds up
the ones that follow it in
the same binary. `allocate_drop_vortex` at 64, 256 and 1024 bytes
improved by 20 to 28% on this
branch, with no change to the code they measure. The run logs list
benchmarks in execution order:
those three run at positions 40 to 42 on develop and 30 to 32 here, and
their immediate
predecessor is the same on both. What is gone is ten earlier benchmarks,
each doing 256
allocate-and-free cycles across five sizes. Two `take_chunked` cases
moved the same way.
The deltas are flat rather than proportional: 14.6, 14.4 and 18.7 µs in
allocation, 39.1 and
40.1 µs in take_chunked. That is the fixed allocator cleanup cost from
#9994 again. Only the
small sizes moved, while 16 KiB and 64 KiB did not, which fits
mimalloc's per-size-class
bookkeeping: these benchmarks were partly measuring the mess their
predecessors left behind.
Checked with clippy under `-D warnings`, the same under `--cfg
codspeed`, and `cargo fmt`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01B6ApyxEXpjiasZbQptsypn
---
_Generated by [Claude Code](https://claude.ai/code)_
---------
Signed-off-by: Claude <[email protected]>
Co-authored-by: Claude <[email protected]>1 parent 21b3dea commit 0fea4a3
7 files changed
Lines changed: 24 additions & 440 deletions
File tree
- vortex-array
- benches
- vortex-buffer/benches
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
149 | 149 | | |
150 | 150 | | |
151 | 151 | | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | 152 | | |
157 | 153 | | |
158 | 154 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | 16 | | |
18 | 17 | | |
19 | 18 | | |
| |||
24 | 23 | | |
25 | 24 | | |
26 | 25 | | |
27 | | - | |
28 | | - | |
29 | 26 | | |
30 | 27 | | |
31 | | - | |
32 | 28 | | |
33 | | - | |
34 | | - | |
35 | 29 | | |
36 | 30 | | |
37 | 31 | | |
| |||
62 | 56 | | |
63 | 57 | | |
64 | 58 | | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | 59 | | |
72 | 60 | | |
73 | 61 | | |
74 | 62 | | |
75 | 63 | | |
76 | 64 | | |
77 | 65 | | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | 66 | | |
96 | 67 | | |
97 | 68 | | |
| |||
120 | 91 | | |
121 | 92 | | |
122 | 93 | | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
174 | 94 | | |
175 | 95 | | |
176 | 96 | | |
| |||
180 | 100 | | |
181 | 101 | | |
182 | 102 | | |
183 | | - | |
184 | | - | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | 103 | | |
190 | 104 | | |
191 | 105 | | |
| |||
267 | 181 | | |
268 | 182 | | |
269 | 183 | | |
270 | | - | |
271 | | - | |
272 | | - | |
273 | | - | |
274 | | - | |
275 | | - | |
276 | | - | |
277 | | - | |
278 | | - | |
279 | | - | |
280 | | - | |
281 | | - | |
282 | | - | |
283 | | - | |
284 | | - | |
285 | | - | |
286 | | - | |
287 | | - | |
288 | | - | |
289 | | - | |
290 | | - | |
291 | | - | |
292 | | - | |
293 | | - | |
294 | | - | |
295 | | - | |
296 | | - | |
297 | | - | |
298 | | - | |
299 | | - | |
300 | | - | |
301 | | - | |
302 | | - | |
303 | | - | |
304 | | - | |
305 | | - | |
306 | | - | |
307 | | - | |
308 | | - | |
309 | | - | |
310 | | - | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | 184 | | |
316 | 185 | | |
317 | 186 | | |
| |||
346 | 215 | | |
347 | 216 | | |
348 | 217 | | |
349 | | - | |
350 | | - | |
351 | | - | |
352 | | - | |
353 | | - | |
354 | | - | |
355 | | - | |
356 | | - | |
357 | | - | |
358 | | - | |
359 | | - | |
360 | | - | |
361 | 218 | | |
362 | 219 | | |
363 | 220 | | |
| |||
368 | 225 | | |
369 | 226 | | |
370 | 227 | | |
371 | | - | |
372 | | - | |
373 | | - | |
374 | | - | |
375 | | - | |
376 | | - | |
377 | | - | |
378 | | - | |
379 | | - | |
380 | | - | |
381 | 228 | | |
382 | 229 | | |
383 | 230 | | |
| |||
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
42 | | - | |
43 | 42 | | |
44 | 43 | | |
45 | 44 | | |
| |||
52 | 51 | | |
53 | 52 | | |
54 | 53 | | |
55 | | - | |
56 | 54 | | |
57 | 55 | | |
58 | 56 | | |
| |||
157 | 155 | | |
158 | 156 | | |
159 | 157 | | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
165 | 162 | | |
166 | 163 | | |
167 | 164 | | |
| |||
259 | 256 | | |
260 | 257 | | |
261 | 258 | | |
262 | | - | |
263 | | - | |
| 259 | + | |
264 | 260 | | |
265 | 261 | | |
266 | 262 | | |
| |||
0 commit comments