Skip to content

Coverage reporting 0% after test reorganization #3004

@EwoutH

Description

@EwoutH

After reorganizing test files into subdirectories, Codecov consistently reports 0% coverage, even when source files are modified.

What's known

Test execution

  • ✅ All 379 tests are discovered correctly
  • ✅ All 379 tests run successfully
  • ✅ Pytest is working as expected

Coverage.py behavior

  • ❌ Coverage.py reports: "No data was collected (no-data-collected)"
  • ❌ Warning from: /opt/hostedtoolcache/Python/3.14.2/x64/lib/python3.14/site-packages/coverage/control.py:957
  • The warning suggests coverage.py ran but didn't measure any lines as executed

Changes made

  1. Test file reorganization (PR Reorganize tests to mirror source module structure #2994):
    • Moved tests from tests/ root into subdirectories:
      • tests/discrete_space/
      • tests/experimental/
      • tests/examples/
      • tests/visualization/
    • Updated import in tests/test_space.py: from tests.discrete_space.test_grid import MockAgent
  2. Configuration changes attempted:
    • Moved omit = ["tests/*"] between [tool.coverage.run] and [tool.coverage.report] - no effect

Current configuration
pyproject.toml:

[tool.coverage.run]
source = ["mesa"]
branch = true
omit = ["tests/*"]

codecov.yaml:

coverage:
  status:
    project:
      default:
        target: 80%
        threshold: 1%

ignore:
  - "benchmarks/**"
  - "mesa/visualization/**"

comment: off

What doesn't explain this

  • ❌ "No source files changed" - we modified source files
  • ❌ "Changed files are ignored" - mesa/experimental/ is not ignored
  • ❌ "Tests aren't running" - all 379 tests execute successfully
  • ❌ "Wrong omit configuration" - tried both configurations

The mystery

Coverage.py is running (we see its warning), pytest is running all tests successfully, but coverage.py claims it measured zero lines. This suggests coverage.py is either:

  1. Not tracing the code during test execution, or
  2. Collecting data but unable to write/report it, or
  3. Running in a mode where it's not hooked into the test execution

Questions to investigate

  1. Is coverage.py actually being invoked during pytest execution in CI?
  2. Are there any pytest plugins or configurations that might interfere with coverage tracing?
  3. Does the test reorganization somehow break coverage.py's ability to trace execution paths?
  4. Is there a mismatch between where coverage.py expects source files vs where they actually are?

Links

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciRelease notes label

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions