Repository navigation
Commit 563f667
Run CodSpeed benchmarks on Python 3.14 and loop the smallest ones (#1498)
<!-- Thank you for your contribution! -->
## What do these changes do?
Moves the CodSpeed benchmark job to Python 3.14, and makes the `repr()`
and view inequality benchmarks repeat their operation 100 times, the way
the other benchmarks already loop. The benchmark job timeout goes from
10 to 15 minutes, since the job takes about 9.5 minutes.
The high "variability" figures on the CodSpeed dashboard (up to about
140%) come from benchmark redefinitions in #1494 and from the real
`add()` and `getall()` performance changes in #1466 and #1484, not from
run-to-run noise. A same-commit rerun on master (242e5bd) reported 266
of 266 benchmarks unchanged. The dashboard window will age out by
itself.
## Are there changes in behavior for the user?
No, only benchmarks and CI change. The reshaped benchmarks, and the move
to 3.14, will cause a one-time jump in their CodSpeed history.
## Is it a substantial burden for the maintainers to support this?
No.
## Related issue number
N/A
## Checklist
- [x] I think the code is well written
- [ ] Unit tests for the changes exist: N/A, this only changes
benchmarks and CI
- [ ] Documentation reflects the changes: N/A
- [ ] If you provide code modification, please add yourself to
`CONTRIBUTORS.txt`: N/A
- [x] Add a new news fragment into the `CHANGES/` folder
- [x] `make doc-spelling` passes and any new technical words are added
to `docs/spelling_wordlist.txt`
<details>
<summary>Agent run details (optional, for reviewers)</summary>
Measurement: valgrind is not available locally, so a small pytest plugin
emulated pytest-codspeed's simulation mode, including the
`gc.collect()`/`gc.disable()` and `random.seed(0)` that
`plugin._measure` does, followed by one warmup call and one measured
call. It recorded user-space retired instructions with
`perf_event_open`, pinned to P-cores. Each run allocated a different
number of dummy objects first, to mimic unrelated heap changes between
commits. The suite ran on CPython 3.15.0rc2 with `PYTHONHASHSEED=0`, 40
runs per variant, using the same seeds for both.
- master benchmarks: every benchmark within 3.6% max/min spread; the
worst was `test_items_view_not_equals[ci-py]`, followed by
`test_values_view_repr[ci-c]` at 2.4%.
- this branch: every benchmark within 3.3% spread; the looped benchmarks
all dropped below 2%.
An earlier revision also added a `gc_disabled` fixture, based on a
harness that did not replicate pytest-codspeed's own GC handling. It was
redundant and has been removed.
Tests (3.14.2): `pytest` passed (2059 passed, 21 skipped); `pytest
--no-c-extensions` passed (1037 passed); `pytest --codspeed
--codspeed-mode simulation` passed (269 benchmarks). 3.15 was tried
first but dropped: setup-python has no prebuilt 3.15 yet, so the job
compiled CPython and ran even longer.
Lint: `pre-commit run` on the changed files passed; `make doc-spelling`
passed.
</details>
Drafted with Claude Code (Opus 5); reviewed by asvetlov.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>1 parent 157c87c commit 563f667
4 files changed
Lines changed: 18 additions & 9 deletions
File tree
- .github/workflows
- CHANGES
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
418 | 418 | | |
419 | 419 | | |
420 | 420 | | |
421 | | - | |
| 421 | + | |
422 | 422 | | |
423 | 423 | | |
424 | 424 | | |
425 | | - | |
| 425 | + | |
426 | 426 | | |
427 | 427 | | |
428 | 428 | | |
429 | | - | |
| 429 | + | |
430 | 430 | | |
431 | 431 | | |
432 | 432 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
602 | 602 | | |
603 | 603 | | |
604 | 604 | | |
605 | | - | |
| 605 | + | |
| 606 | + | |
606 | 607 | | |
607 | 608 | | |
608 | 609 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
| 27 | + | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| |||
133 | 134 | | |
134 | 135 | | |
135 | 136 | | |
136 | | - | |
| 137 | + | |
| 138 | + | |
137 | 139 | | |
138 | 140 | | |
139 | 141 | | |
| |||
155 | 157 | | |
156 | 158 | | |
157 | 159 | | |
158 | | - | |
| 160 | + | |
| 161 | + | |
159 | 162 | | |
160 | 163 | | |
161 | 164 | | |
| |||
264 | 267 | | |
265 | 268 | | |
266 | 269 | | |
267 | | - | |
| 270 | + | |
| 271 | + | |
268 | 272 | | |
269 | 273 | | |
270 | 274 | | |
| |||
274 | 278 | | |
275 | 279 | | |
276 | 280 | | |
277 | | - | |
| 281 | + | |
| 282 | + | |
0 commit comments