Fix the inert integration timeout opt-in and give the RAG ask tests headroom#546
Merged
Conversation
The macos integration leg intermittently blew the 180s budget on test_ask_known_fact with the test itself passing: a full sync plus a RAG ask is the heaviest work in the suite and a slow virtualized runner spent the whole budget on it, the timer landing in teardown. The conftest's documented escape hatch (opt in with pytest.mark.timeout) turned out to be inert: add_marker prepends, so the injected default was always the closest marker and overrode every per-test opt-in. Proven by a probe test sleeping past a class-level cap without dying. The hook now only defaults items that carry no timeout of their own, and TestAsk opts in to 300s; a genuine hang still dies well short of the job timeout.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The macos integration leg intermittently failed test_ask_known_fact with a 180s pytest-timeout, the test itself passing: a full sync plus a RAG ask is the heaviest work in the suite, and a slow virtualized runner spent the entire budget on it.
Worse, the conftest's documented escape hatch (opt in with pytest.mark.timeout) has been inert all along: add_marker prepends, so the injected 180s default was always the closest marker and silently overrode every per-test opt-in. A probe test sleeping past a class-level cap survived to prove it.
Solution
The collection hook now only defaults items that carry no timeout marker of their own, which makes the opt-in real, and TestAsk opts in to 300 seconds. Unmarked tests keep the 180s default; a genuine hang still dies well short of the job timeout.