Skip to content

CI: test_delete flaky on macOS 3.11 #202

@matin

Description

@matin

Description

VCR cassette tests intermittently fail in CI by hitting the real Garmin API instead of playing back recorded cassettes. This has been seen on multiple runners:

  • test_delete on macOS 3.11 (401 from real API)
  • test_login_command on Windows 3.13 (401 from real API)

Root cause

The VCR fixture in tests/conftest.py switches between record_mode="none" (playback only) and the default "once" (record if no cassette) based on whether GARTH_HOME is set:

@pytest.fixture
def vcr(vcr):
    if "GARTH_HOME" not in os.environ:
        vcr.record_mode = "none"
    return vcr

When GARTH_HOME is set (or leaks from the environment), VCR uses "once" mode. If a cassette doesn't match the request (e.g. stale auth headers), VCR falls through to the real API, which returns 401 since the test tokens are fake.

The real API should never be called in CI. The current logic is backwards — it should default to "none" (playback only) and only allow recording when explicitly opted in.

Fix

  1. Always use record_mode="none" in CI — VCR should never hit the real API during test runs
  2. Use an explicit env var for recording (e.g. GARTH_RECORD_CASSETTES=true) instead of piggy-backing on GARTH_HOME
  3. The _disable_telemetry autouse fixture should also clear GARTH_HOME to prevent env var leaks from affecting VCR mode

Proposed conftest change:

@pytest.fixture(autouse=True)
def _clean_env(monkeypatch):
    """Disable telemetry and prevent env leaks in all tests."""
    monkeypatch.setenv("GARTH_TELEMETRY_ENABLED", "false")
    monkeypatch.delenv("GARTH_HOME", raising=False)
    monkeypatch.delenv("GARTH_TOKEN", raising=False)

@pytest.fixture
def vcr(vcr):
    if os.environ.get("GARTH_RECORD_CASSETTES") != "true":
        vcr.record_mode = "none"
    return vcr

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions