Skip to content

Mutation testing cannot attribute mutants for any scripts/ module (synthetic module names in tests_py/scripts) #262

Closed

Description

@cdeust

What

scripts/mutation_check.sh cannot mutate anything under scripts/. The run
aborts before producing a result:

$ scripts/mutation_check.sh tests_py/scripts/test_launcher_deps.py scripts/launcher_deps.py
Stopping early, because tests recorded trampoline hits but none match any mutant key.
It looks like tests import the source under a different module path than mutmut sees
from the file path.
Recorded keys (e.g.): ['_cortex_launcher_deps.x__importable', ...]
Expected keys (e.g.): ['scripts.launcher_deps.x__importable', ...]

Cause: every test module under tests_py/scripts/ loads its subject with
importlib.util.spec_from_file_location(...) under a synthetic module name
(_cortex_launcher_deps, and the same pattern in
test_generate_pip_constraints.py), while mutmut keys mutants by the path-derived
name scripts.launcher_deps. The trampoline is therefore never matched.

The path-import is deliberate for launcher_deps.py itself — it is loaded by
scripts/launcher.py before the plugin's dependencies exist on sys.path, so it
must stay importable by path at runtime. Nothing forces the TESTS to load it that
way, but changing them is a test-infrastructure change with its own blast radius,
not a line in a feature PR.

Consequence: §12's per-commit mutation gate is silently inapplicable to the entire
scripts/ tree — which holds the launcher's dependency bootstrap, the pip
constraint generator, the doc-claim gate and the badge generators. mutmut fails
loudly here (good), but the gate is simply absent for those modules.

Found while adding python-dateutil to _BASE_PACKAGES in #252 / PR #259 — the
scoped mutation run for that change could not execute.

Acceptance criteria

  1. scripts/mutation_check.sh <test> <scripts/...> produces a real mutant tally
    (killed/survived) for at least scripts/launcher_deps.py and
    scripts/generate_pip_constraints.py.
  2. launcher_deps.py remains loadable by path at runtime (the launcher's
    pre-dependency bootstrap must not regress) — assert it, don't assume it.
  3. Any repository file a tests_py/scripts/ test reads (uv.lock, the doc-claim
    sources) is listed in [tool.mutmut] also_copy, so mutants fail on the
    mutation and not on FileNotFoundError.
  4. Surviving mutants in the touched modules are triaged per §12.4: killed, or
    documented equivalents.

Activity

  1. added 12 commits that reference this issue on Jul 30, 2026
    7d2842d
    a50b912
    5a3c010
    9a7528c
    a3c54cd
    05c7169
    886b2dd
    ee822a9
    8883188
    822ac35
    4f12c05
    3d056be
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions