Skip to content

Pytest drops tests/web_admin conftest fixtures when a web_admin file is revisited non-adjacently in the argument list #273

Closed

Description

@rcbevans

Confirmed in the independent review of PR #262 with controlled runs (pre-existing test-infra broken window, not introduced by any PR in this effort).

Repro: uv run --no-sync pytest tests/web_admin/test_actor_stats.py tests/test_release_breaking_change_accuracy.py tests/web_admin/test_jobs.py → 59 passed, 28 ERRORS: every test_jobs.py test fails at setup with fixture 'stub_pool' not found / fixture 'make_app' not found; the available-fixture list contains the root tests/conftest.py fixtures but none of tests/web_admin/conftest.py's, including the autouse _dev_env.

Controls (all clean): single web_admin file; root-file-first ordering (pytest tests/test_release_breaking_change_accuracy.py tests/web_admin/test_actor_stats.py); three adjacent web_admin files (119 passed). The trigger is a tests/web_admin/ directory revisited non-adjacently in the argument list; pytest's conftest collection appears to cache/skip on directory revisits, silently gutting the revisited file's fixtures.

Practical consequence: any mixed command line of that shape fails in ways that look like code breakage, not harness breakage; it bit two independent agents' test runs in this effort before it was identified. make test/make test-fast (directory-scoped) are unaffected.

Filed per this effort's policy of tracking every finding. Direction: investigate pytest's conftest collection on arg-order directory revisits (likely a collection-cache interaction); a root-conftest assertion or a documented harness rule could pin it.

Activity

  1. self-assigned this
    on Sep 17, 2026
  2. rcbevans commented on Sep 18, 2026

    @rcbevans
    ContributorAuthor

    Reproduced on main (f75f752): the exact command from the issue gives 28 setup errors, all 'fixture stub_pool not found' / 'make_app not found' in the revisited file, while the root-file-first ordering of the same files passes clean. Still present, still a trap for any mixed command line.

    Cheapest durable guard without waiting on pytest upstream: a root-conftest collection check (or a small session-scoped test) that runs a two-file argument list revisiting tests/web_admin non-adjacently and asserts the web_admin fixtures resolve, so any regression here fails loudly in CI instead of in someone's debugging session. Pair it with a one-line harness rule in the test docs. The repo's pytest pin should also be checked against upstream conftest-collection fixes in case a version bump dissolves the whole problem.

  3. added a commit that references this issue on Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions