Repository navigation
Description
Problem (one or two sentences)
The required e2e-mock CI check fails intermittently. One test times out after 30s while 85 or more tests in the same run pass.
Context (who is affected and when)
Every PR that runs the e2e.yml workflow is affected. A transient runner-network stall during the mock run fails the whole job and holds the PR review gate. Three unrelated branches hit this on 2026-10-13:
- PR Fix interrupted child redelegation #1905, run 38343639461, branch
fix/interrupted-child-redelegation, testMarkdown List Rendering > Should render unordered lists with bullets in chat - Run 38391536768, branch
vps2/f6-e2e-viewstate, testRoo Code View State > sidebar and tab panel keep mode isolated through the real ContextProxy singleton - Run 38388064934, same branch and test, failed again 30 minutes later
All three branches passed on later runs with no related code change. The PR #1905 commit touches no fetcher, model-cache, or e2e harness code, and the same test passed in 534ms at the prior head in run 38329310237.
Reproduction steps
Not reproducible on demand. The pattern from the failing runs:
- Mock e2e starts and the extension host activates.
- The host runs background model-catalog refreshes against real endpoints (OpenRouter, Vercel AI Gateway, NanoGPT).
- A runner-network stall makes those fetches hang. Each fetch holds a socket until the 15s abort timer fires (
MODEL_CATALOG_FETCH_TIMEOUT_MS = 15_000). A second wave follows immediately, so the stall window reaches about 30s. - The test task in progress never issues its first API request during this window.
waitUntilCompletedrejects withTimeout after 30s.
Evidence from the PR #1905 run job log:
- 03:52:48 test task starts, checkpoints initialize.
- 03:52:49 to 03:53:02: no task-loop activity at all.
- 03:53:02 and 03:53:17: two waves of
[getModels] ... AbortError: This operation was abortedfor openrouter, plusrequestRouterModelsfailures. 16 hang-abort lines in the failing run, zero in the passing run. - 03:53:18:
Timeout after 30sfor the test. The two sibling tests then pass in 768ms and 1273ms with the identical start-and-wait flow.
Expected result
Mock e2e never depends on the real network. A transient runner-network stall must not fail the required check.
Actual result
Real-network catalog fetches run during mock e2e. When they hang, they block task startup for about 30s, one test fails, and the whole required job fails.
Variations tried
Re-run of the failed job. The PR #1905 re-run was started manually; no code change was involved.
App Version
CI only. e2e-mock job in .github/workflows/e2e.yml, GitHub-hosted ubuntu-latest runners.
Relevant logs or errors
Error: Timeout after 30s
at Timeout._onTimeout (out/suite/utils.js:15:20)
[getModels] Failed to fetch models in modelCache for openrouter: AbortError: This operation was aborted
at Timeout._onTimeout (node:internal/abort_controller:127:7)
Suggested fixes
- Extend the mock fast-fallback to all real-network catalog fetchers under mock e2e. Today only the DeepSeek fetcher honors
E2E_MOCK_MODEL_LIST_FALLBACK. OpenRouter, Vercel AI Gateway, and NanoGPT have no equivalent. - Alternatively, add
aimockfixtures for the catalog endpoints so mock runs never touch the real network. - Optionally, add one retry to the e2e poll helper or raise its 30s ceiling, so a single stall cannot fail the required check.
Activity
edelauna commentedon Oct 13, 2026
Provenance check after the initial report:
Timeout after 30sflake predates PR refactor: Fetch model lists via ApiHandler interface #1906. Run 38289267945 failed with the identical error in the samewaitUntilCompletedhelper on 2026-10-12 18:11Z, before refactor: Fetch model lists via ApiHandler interface #1906 merged at 20:01Z. That pre-merge run shows zero model-fetch aborts, so its stall source was different.refactor: Fetch model lists via ApiHandler interface, authorp12tic) centralized catalog fetching inmodelCache.tswith the 15s timeout and background refresh. Every post-merge failure shows the OpenRouter abort-wave signature (16 hang-abort lines in run 38343639461); the pre-merge failure shows none.Conclusion: the base fragility (single-shot 30s wait, real-network dependence during mock e2e) predates #1906. The refactor did not introduce the flake, but it added a background real-network fetch fan-out that widens the stall window to the full 30s budget when the runner network hiccups. Failure counts in this sample: 1 timeout flake in 100 runs before the merge, 3 timeout flakes in one day after it. The sample is small, so treat the frequency shift as indicative, not proven.
The suggested fixes stay valid. Extending the mock fallback or stubbing catalog endpoints removes the new stall source; a retry in the poll helper covers the pre-existing one.