CodSpeed Performance Report

Merging #12314 will not alter performance

_{Comparing 12313-test_manholepy-is-failing-with-latest-313-pre-release (5a8d3d5) with trunk (f0afc5e)}

Summary

✅ 23 untouched benchmarks

I've been seeing what I think is the same failure since updating to Python 3.12.6.

It's possible this is a regression in CPython. I'll try to reproduce; perhaps a better way to implement it is to inspect the possibly-skipped frame and see if it should be skipped, given it's in a released version. And I guess file a bug report.

Thanks for the PR

I see that on trunk, these tests were not failing with 3.13.0rc1 (main, Aug 29 2024, 14:16:22) [GCC 11.4.0]

https://github.com/twisted/twisted/actions/runs/10774970649/job/29878322696#step:10:1079

I did a re-run to get latest 3.13.0rc2 (main, Sep 9 2024, 03:11:37) [GCC 11.4.0] and I can see that trunk tests are failing

https://github.com/twisted/twisted/actions/runs/10774970649/job/30162695096#step:10:1081

There's a similar failure with Debian's Python 3.12.6, so maybe "less than 3.13" isn't quite the right test?

Thanks for the feedback

It looks like via actions/setup-python@v5 we only get CPython (3.12.5)

Maybe we need to wait a bit more. I see that 3.12.6 is available via actions/python-versions and with a definition of 3.12 , actions/setup-python should have picked up the latest release ... so 3.12.6

https://github.com/actions/python-versions/releases/tag/3.12.6-10765725458

The strange thing, is that on the actions/setup-python CI, I can see that 3.12 is picked up as 3.12.6
https://github.com/actions/setup-python/actions/runs/10795247111/job/29941593380#step:3:3

I bisected Python v3.12.5..v3.12.6 and got python/cpython@8fe586d, i.e. python/cpython#122478. Does that help with working out the correct fix?

To replicate this error on 3.12.6 we need

Run actions/setup-python@v5
  with:
    python-version: 3.12
-    check-latest: false
+    check-latest: true

I have pushed a change on this PR to reproduce the error in 3.12.6

I updated the check to be tied to the symptoms, rather than version specific.