Flaky / failing test on main: simple-completion-runtime.test.ts > prepareSimpleCompletionModel > can preserve asynchronous provider model discovery
Summary
The test added in commit 25ca39e876 ("fix(tts): preserve async model discovery") is failing on the checks-node-agentic-agents-core-models and checks-node-agentic-agents-core-runtime lanes — including on the commit itself as it landed on main. This is causing every PR that picks up the merge from main to inherit two red lanes through no fault of their own.
Evidence
Commit on main (when it landed): run 27319176413, job 80706378249 — checks-node-agentic-agents-core-runtime ❌ FAILURE at 2026-06-11T02:12:18Z, on commit 25ca39e87622b04f1f24ffddb0317481a2f13aa8.
Re-observed today on PR #90579 head dd58c51ca1 (run 27330511527, job 80741593581) — same lane failing, same test, same error shape:
FAIL src/agents/simple-completion-runtime.test.ts > prepareSimpleCompletionModel > can preserve asynchronous provider model discovery
AssertionError: expected "vi.fn()" to not be called at all, but actually been called 1 times
1st vi.fn() call: ["anthropic", "claude-opus-4-6", undefined, undefined, { authProfileId: undefined, preferredProfile: undefined }]
src/agents/simple-completion-runtime.test.ts:488:42
PR #90579 only touches src/media/* — no src/agents/* changes — so this is reproducing the main-lane failure, not introducing it.
What the test is asserting
expect(hoisted.resolveModelMock).not.toHaveBeenCalled();
expect(hoisted.resolveModelAsyncMock).toHaveBeenCalledWith(
"anthropic", "claude-opus-4-6", undefined, undefined, {},
);
The useAsyncModelResolution: true path is supposed to skip the sync resolveModel and only invoke resolveModelAsync. The failure shows resolveModelMock is being invoked once with "anthropic", "claude-opus-4-6", suggesting either:
- A separate code path inside
prepareSimpleCompletionModel still falls through to resolveModel even when useAsyncModelResolution: true, OR
- Mock state from a prior test in the file is leaking into this one (the file uses
vi.hoisted for the mocks; if cleanup between tests isn't resetting call state, a previous prepareSimpleCompletionModel invocation could be the source of the recorded call)
The 1st-call payload shape ({authProfileId, preferredProfile} style) doesn't quite match the test's own arguments (it passes cfg: undefined, no profile fields), so the leaking-mock-state hypothesis is the more likely one — but I haven't tried to reproduce locally.
Impact
Currently affecting at least PR #90579; almost certainly affecting other PRs that have rebased onto / merged in main since 2026-06-11T02:12Z. Two CI lanes red on every affected PR until this is resolved.
Suggested follow-up
A short repro / fix from @vincentkoc would be quickest since the test was added in the same commit as the production change. If it's mock-state leakage, an explicit vi.clearAllMocks() in a beforeEach (or per-test reset of just the hoisted mocks) is usually enough. If it's a genuine code-path issue, the assertion expect(hoisted.resolveModelMock).not.toHaveBeenCalled() is correctly catching it and the fix is in prepareSimpleCompletionModel.
cc @steipete (PR review queue context)
Flaky / failing test on
main:simple-completion-runtime.test.ts > prepareSimpleCompletionModel > can preserve asynchronous provider model discoverySummary
The test added in commit 25ca39e876 ("fix(tts): preserve async model discovery") is failing on the
checks-node-agentic-agents-core-modelsandchecks-node-agentic-agents-core-runtimelanes — including on the commit itself as it landed onmain. This is causing every PR that picks up the merge frommainto inherit two red lanes through no fault of their own.Evidence
Commit on main (when it landed): run 27319176413, job 80706378249 —
checks-node-agentic-agents-core-runtime❌ FAILURE at2026-06-11T02:12:18Z, on commit25ca39e87622b04f1f24ffddb0317481a2f13aa8.Re-observed today on PR #90579 head
dd58c51ca1(run 27330511527, job 80741593581) — same lane failing, same test, same error shape:PR #90579 only touches
src/media/*— nosrc/agents/*changes — so this is reproducing the main-lane failure, not introducing it.What the test is asserting
The
useAsyncModelResolution: truepath is supposed to skip the syncresolveModeland only invokeresolveModelAsync. The failure showsresolveModelMockis being invoked once with"anthropic","claude-opus-4-6", suggesting either:prepareSimpleCompletionModelstill falls through toresolveModeleven whenuseAsyncModelResolution: true, ORvi.hoistedfor the mocks; if cleanup between tests isn't resetting call state, a previousprepareSimpleCompletionModelinvocation could be the source of the recorded call)The 1st-call payload shape (
{authProfileId, preferredProfile}style) doesn't quite match the test's own arguments (it passescfg: undefined, no profile fields), so the leaking-mock-state hypothesis is the more likely one — but I haven't tried to reproduce locally.Impact
Currently affecting at least PR #90579; almost certainly affecting other PRs that have rebased onto / merged in
mainsince2026-06-11T02:12Z. Two CI lanes red on every affected PR until this is resolved.Suggested follow-up
A short repro / fix from @vincentkoc would be quickest since the test was added in the same commit as the production change. If it's mock-state leakage, an explicit
vi.clearAllMocks()in abeforeEach(or per-test reset of just the hoisted mocks) is usually enough. If it's a genuine code-path issue, the assertionexpect(hoisted.resolveModelMock).not.toHaveBeenCalled()is correctly catching it and the fix is inprepareSimpleCompletionModel.cc @steipete (PR review queue context)