Repository navigation
test: raise the pytest floor to 9.2, which fixes the skipped conftest fixtures behind a flaky MCP auth test - #3880
Merged
Merged
Conversation
…lection's order cannot skip it (GH-3859) `test_mcp_auth_tool_answers_as_the_route_does` failed in a selection that names `tests/ui` files on both sides of a file outside it. pytest 9 ties the autouse fixtures of a sub-directory's conftest to the directory node it loaded the conftest for; the second node it makes for `tests/ui` has none, so `_forget_auth_probes` of `tests/ui/conftest.py` did not run for `test_core.py` and `test_restapi.py`. The metno/frost probe that `test_coverage_and_auth_tell_a_credential_that_could_not_be_checked_as_null` leaves in the cache (a `valid: False`, kept 60 s by `get_auth`) then answered the MCP test, which expected the probe it had stubbed. The fixture moves to `tests/conftest.py`, whose autouse fixtures reach every test, and clears only where `wetterdienst.ui.core` was imported. A test runs pytest's `--setup-plan` over the order that failed and asks every test for the fixture. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0187kUCUFKYgEkY8mDbH23V8
❌ 5 Tests Failed:
View the top 1 failed test(s) by shortest run time
View the full list of 4 ❄️ flaky test(s)
To view more test analytics, go to the Test Analytics Dashboard |
…back (GH-3859) `test_mcp_auth_tool_answers_as_the_route_does` failed in a selection that names `tests/ui` files on both sides of a file outside it. That is a regression of pytest 9.1.0 and 9.1.1 (pytest-dev/pytest#14635, "fixtures not found when a parent directory appears multiple times in a argument list"), fixed in 9.2.0 (pytest-dev/pytest#14645): the autouse fixtures of a sub-directory's conftest did not reach the tests of the second node pytest made for that directory, so `_forget_auth_probes` of `tests/ui/conftest.py` did not run for `test_core.py` and `test_restapi.py`, and the probe that an earlier test left in the cache answered the MCP test. The floor of the dev group is `pytest>=9.2` instead (uv.lock: pytest 9.1.1 to 9.2.0 only). `tests/ui/conftest.py` and its fixture are as on main, and the workaround of the first commit of this branch, with its guard test, is gone. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0187kUCUFKYgEkY8mDbH23V8
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3859
Cause
test_mcp_auth_tool_answers_as_the_route_doesfailed in a selection that namestests/uifiles on both sides of a file outsidetests/ui. That is a regression of pytest 9.1.0 and 9.1.1, pytest-dev/pytest#14635 ("fixtures not found when a parent directory appears multiple times in a argument list"), fixed by #14645 in 9.2.0.tests/ui, which had none of the autouse fixtures oftests/ui/conftest.py. So_forget_auth_probesdid not run fortest_core.pyandtest_restapi.py.test_coverage_and_auth_tell_a_credential_that_could_not_be_checked_as_nullleaves incore._auth_probes(avalid: False, kept 60 s byget_auth) then answered the MCP test. The library cache is keyed correctly and is not at fault.tests/provider/conftest.pyandtests/provider/metno/frost/conftest.pyin such selections.Fix
The floor of the dev group is
pytest>=9.2(one higher minimum, no workaround).uv.lockchanges only pytest 9.1.1 to 9.2.0.tests/ui/conftest.pystays as on main. The diff to main ispyproject.tomlanduv.lock. pytest is in the dev group, not an extra, so there is no CHANGELOG entry.Run (pytest 9.2.0)
-n 4, 3 times: 1523 passed, 6 skipped each time. On 9.1.1 it failed, 3 tests.pytest tests/ui/test_empty_listing.py tests/test_exceptions.py tests/ui/test_core.py --setup-plan:_forget_auth_probeson all 135 tests oftest_core.py.pytest tests/provider/metno/frost/test_is_valid.py tests/test_info.py tests/provider/metno/frost/test_values_failures.py tests/provider/dwd/dmo/test_api.py --setup-plan: thetests/providerautouse fixtures (recorded_frames,_timestamp_column_is_utc_microseconds) and themetno/frostone (_no_second_look_at_a_refusal) on every test of those directories.uv run poe format,lint,typecheck: clean.uv run poe "test:offline": 13990 passed, 49 skipped.🤖 Generated with Claude Code
https://claude.ai/code/session_0187kUCUFKYgEkY8mDbH23V8