-
Notifications
You must be signed in to change notification settings - Fork 65
CI: test_delete flaky on macOS 3.11 #202
Description
Description
VCR cassette tests intermittently fail in CI by hitting the real Garmin API instead of playing back recorded cassettes. This has been seen on multiple runners:
test_deleteon macOS 3.11 (401 from real API)test_login_commandon Windows 3.13 (401 from real API)
Root cause
The VCR fixture in tests/conftest.py switches between record_mode="none" (playback only) and the default "once" (record if no cassette) based on whether GARTH_HOME is set:
@pytest.fixture
def vcr(vcr):
if "GARTH_HOME" not in os.environ:
vcr.record_mode = "none"
return vcrWhen GARTH_HOME is set (or leaks from the environment), VCR uses "once" mode. If a cassette doesn't match the request (e.g. stale auth headers), VCR falls through to the real API, which returns 401 since the test tokens are fake.
The real API should never be called in CI. The current logic is backwards — it should default to "none" (playback only) and only allow recording when explicitly opted in.
Fix
- Always use
record_mode="none"in CI — VCR should never hit the real API during test runs - Use an explicit env var for recording (e.g.
GARTH_RECORD_CASSETTES=true) instead of piggy-backing onGARTH_HOME - The
_disable_telemetryautouse fixture should also clearGARTH_HOMEto prevent env var leaks from affecting VCR mode
Proposed conftest change:
@pytest.fixture(autouse=True)
def _clean_env(monkeypatch):
"""Disable telemetry and prevent env leaks in all tests."""
monkeypatch.setenv("GARTH_TELEMETRY_ENABLED", "false")
monkeypatch.delenv("GARTH_HOME", raising=False)
monkeypatch.delenv("GARTH_TOKEN", raising=False)
@pytest.fixture
def vcr(vcr):
if os.environ.get("GARTH_RECORD_CASSETTES") != "true":
vcr.record_mode = "none"
return vcr