Skip to content

TST: work around flaky test on free-threaded build#26503

Merged
charris merged 1 commit intonumpy:mainfrom
ngoldbaum:fix-free-threaded-test-flake
May 23, 2024
Merged

TST: work around flaky test on free-threaded build#26503
charris merged 1 commit intonumpy:mainfrom
ngoldbaum:fix-free-threaded-test-flake

Conversation

@ngoldbaum
Copy link
Member

The newly added free-threaded CI is seeing intermittent test failures caused by this test. See e.g. this run.

This is a weird bug, from the error message, somehow a RuntimeWarning is getting created, and then during the handling of the warning, pytest doesn't see it. But only sometimes. In my test using pytest-repeat to repeatedly run this test I see unhandled warnings anywhere from 0% to 20% of the time. It happens more often in an optimized build of numpy than in a debug build (which is why I missed it during local testing).

This smells like a bug in either pytest or CPython and I will try to figure out what exactly is happening. For now it looks like pytest.raises doesn't have the same issue.

@ngoldbaum ngoldbaum added the 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703) label May 22, 2024
@ngoldbaum
Copy link
Member Author

I ended up tracing this to the fact that warnings.catch_warnings() is not thread safe so pytest.warns isn't either: pytest-dev/pytest#12356.

@charris charris merged commit 6c648e7 into numpy:main May 23, 2024
@charris
Copy link
Member

charris commented May 23, 2024

Let's give it a shot. Thanks Nathan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

05 - Testing 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants