Skip to content

Commit 7014615

Browse files
d-v-bclaude
andcommitted
ci: install just as a binary in the benchmark job
Adding `actions/setup-python` to this job so that `pip install rust-just` had an interpreter also changed which interpreter hatch built `test.py3.12-minimal` on: the runner's own Python was replaced by /opt/hostedtoolcache/Python/3.12.5/arm64. CodSpeed measured the result as a 12.97% degradation across 26 benchmarks, spread -12.76% to -13.99% -- a range of 1.2 points over slice indexing, sharded Morton indexing, three array shapes and two store types. Nothing in this branch touches src/, tests/benchmarks/ or packages/; that uniformity is an interpreter swap, not a regression, and CodSpeed flagged it as "different runtime environments detected". Install just with extractions/setup-just instead, matching the three package workflows. No interpreter is set up, so hatch resolves the same Python the baseline used, and `just-version` is pinned to satisfy zizmor's unpinned-tools audit. The benchmarks need a re-run to produce a comparison against a matching environment; the numbers on the previous run should not be acknowledged as a regression. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <[email protected]>
1 parent 45da006 commit 7014615

1 file changed

Lines changed: 8 additions & 5 deletions

File tree

‎.github/workflows/codspeed.yml‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,16 +27,19 @@ jobs:
2727
with:
2828
fetch-depth: 0
2929
persist-credentials: false
30-
- name: Set up Python
31-
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
32-
with:
33-
python-version: '3.12'
3430
- name: Install Hatch
3531
uses: pypa/hatch@f647ed70d49adb885f53a27d1c7f5bdaeacf2c60
3632
with:
3733
version: '1.16.5'
34+
# Installed as a binary rather than with `pip install rust-just`, which would
35+
# need an interpreter set up in this job. Benchmarks are only comparable across
36+
# runs if the interpreter underneath them does not move: pinning a Python here
37+
# measured a uniform ~13% slowdown on every benchmark, because hatch then built
38+
# the environment on that interpreter instead of the runner's own.
3839
- name: Install just
39-
run: python -m pip install rust-just==1.58.0
40+
uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
41+
with:
42+
just-version: 1.58.0
4043
- name: Run the benchmarks
4144
uses: CodSpeedHQ/action@373d6868929f444bc08d901fd0eb0ad52a8875ea # v5.2.1
4245
env:

0 commit comments

Comments
 (0)