Skip to content

Commit 563f667

Browse files
Run CodSpeed benchmarks on Python 3.14 and loop the smallest ones (#1498)
<!-- Thank you for your contribution! --> ## What do these changes do? Moves the CodSpeed benchmark job to Python 3.14, and makes the `repr()` and view inequality benchmarks repeat their operation 100 times, the way the other benchmarks already loop. The benchmark job timeout goes from 10 to 15 minutes, since the job takes about 9.5 minutes. The high "variability" figures on the CodSpeed dashboard (up to about 140%) come from benchmark redefinitions in #1494 and from the real `add()` and `getall()` performance changes in #1466 and #1484, not from run-to-run noise. A same-commit rerun on master (242e5bd) reported 266 of 266 benchmarks unchanged. The dashboard window will age out by itself. ## Are there changes in behavior for the user? No, only benchmarks and CI change. The reshaped benchmarks, and the move to 3.14, will cause a one-time jump in their CodSpeed history. ## Is it a substantial burden for the maintainers to support this? No. ## Related issue number N/A ## Checklist - [x] I think the code is well written - [ ] Unit tests for the changes exist: N/A, this only changes benchmarks and CI - [ ] Documentation reflects the changes: N/A - [ ] If you provide code modification, please add yourself to `CONTRIBUTORS.txt`: N/A - [x] Add a new news fragment into the `CHANGES/` folder - [x] `make doc-spelling` passes and any new technical words are added to `docs/spelling_wordlist.txt` <details> <summary>Agent run details (optional, for reviewers)</summary> Measurement: valgrind is not available locally, so a small pytest plugin emulated pytest-codspeed's simulation mode, including the `gc.collect()`/`gc.disable()` and `random.seed(0)` that `plugin._measure` does, followed by one warmup call and one measured call. It recorded user-space retired instructions with `perf_event_open`, pinned to P-cores. Each run allocated a different number of dummy objects first, to mimic unrelated heap changes between commits. The suite ran on CPython 3.15.0rc2 with `PYTHONHASHSEED=0`, 40 runs per variant, using the same seeds for both. - master benchmarks: every benchmark within 3.6% max/min spread; the worst was `test_items_view_not_equals[ci-py]`, followed by `test_values_view_repr[ci-c]` at 2.4%. - this branch: every benchmark within 3.3% spread; the looped benchmarks all dropped below 2%. An earlier revision also added a `gc_disabled` fixture, based on a harness that did not replicate pytest-codspeed's own GC handling. It was redundant and has been removed. Tests (3.14.2): `pytest` passed (2059 passed, 21 skipped); `pytest --no-c-extensions` passed (1037 passed); `pytest --codspeed --codspeed-mode simulation` passed (269 benchmarks). 3.15 was tried first but dropped: setup-python has no prebuilt 3.15 yet, so the job compiled CPython and ran even longer. Lint: `pre-commit run` on the changed files passed; `make doc-spelling` passed. </details> Drafted with Claude Code (Opus 5); reviewed by asvetlov. --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
1 parent 157c87c commit 563f667

4 files changed

Lines changed: 18 additions & 9 deletions

File tree

‎.github/workflows/ci-cd.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -418,15 +418,15 @@ jobs:
418418
needs:
419419
- pre-setup # transitive, for accessing settings
420420
runs-on: ubuntu-latest
421-
timeout-minutes: 10
421+
timeout-minutes: 15
422422
steps:
423423
- name: Checkout project
424424
uses: actions/checkout@v7
425-
- name: Setup Python 3.13
425+
- name: Setup Python 3.14
426426
id: python-install
427427
uses: actions/setup-python@v7
428428
with:
429-
python-version: 3.13
429+
python-version: 3.14
430430
cache: pip
431431
cache-dependency-path: requirements/*.txt
432432
- name: Install dependencies

‎CHANGES/1498.contrib.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
The ``repr()`` and view inequality benchmarks were updated to repeat their
2+
operation in a loop, like the other benchmarks, and the CodSpeed benchmark
3+
job was moved to Python 3.14 -- by :user:`asvetlov`.

‎tests/test_multidict_benchmarks.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -602,7 +602,8 @@ def test_multidict_repr(
602602

603603
@benchmark
604604
def _run() -> None:
605-
repr(md)
605+
for _ in range(100):
606+
repr(md)
606607

607608

608609
def test_create_empty_multidict(

‎tests/test_views_benchmarks.py‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,8 @@ def test_keys_view_not_equals(
2424

2525
@benchmark
2626
def _run() -> None:
27-
assert md1.keys() != md2.keys()
27+
for _ in range(100):
28+
assert md1.keys() != md2.keys()
2829

2930

3031
def test_keys_view_more(
@@ -133,7 +134,8 @@ def test_keys_view_repr(
133134

134135
@benchmark
135136
def _run() -> None:
136-
repr(md.keys())
137+
for _ in range(100):
138+
repr(md.keys())
137139

138140

139141
def test_items_view_equals(
@@ -155,7 +157,8 @@ def test_items_view_not_equals(
155157

156158
@benchmark
157159
def _run() -> None:
158-
assert md1.items() != md2.items()
160+
for _ in range(100):
161+
assert md1.items() != md2.items()
159162

160163

161164
def test_items_view_more(
@@ -264,7 +267,8 @@ def test_items_view_repr(
264267

265268
@benchmark
266269
def _run() -> None:
267-
repr(md.items())
270+
for _ in range(100):
271+
repr(md.items())
268272

269273

270274
def test_values_view_repr(
@@ -274,4 +278,5 @@ def test_values_view_repr(
274278

275279
@benchmark
276280
def _run() -> None:
277-
repr(md.values())
281+
for _ in range(100):
282+
repr(md.values())

0 commit comments

Comments
 (0)