Skip to content

Flaky / failing: simple-completion-runtime > can preserve asynchronous provider model discovery (introduced 25ca39e876) #92117

Description

@wangwllu

Flaky / failing test on main: simple-completion-runtime.test.ts > prepareSimpleCompletionModel > can preserve asynchronous provider model discovery

Summary

The test added in commit 25ca39e876 ("fix(tts): preserve async model discovery") is failing on the checks-node-agentic-agents-core-models and checks-node-agentic-agents-core-runtime lanes — including on the commit itself as it landed on main. This is causing every PR that picks up the merge from main to inherit two red lanes through no fault of their own.

Evidence

Commit on main (when it landed): run 27319176413, job 80706378249checks-node-agentic-agents-core-runtime ❌ FAILURE at 2026-06-11T02:12:18Z, on commit 25ca39e87622b04f1f24ffddb0317481a2f13aa8.

Re-observed today on PR #90579 head dd58c51ca1 (run 27330511527, job 80741593581) — same lane failing, same test, same error shape:

FAIL src/agents/simple-completion-runtime.test.ts > prepareSimpleCompletionModel > can preserve asynchronous provider model discovery
AssertionError: expected "vi.fn()" to not be called at all, but actually been called 1 times
1st vi.fn() call: ["anthropic", "claude-opus-4-6", undefined, undefined, { authProfileId: undefined, preferredProfile: undefined }]
   src/agents/simple-completion-runtime.test.ts:488:42

PR #90579 only touches src/media/* — no src/agents/* changes — so this is reproducing the main-lane failure, not introducing it.

What the test is asserting

expect(hoisted.resolveModelMock).not.toHaveBeenCalled();
expect(hoisted.resolveModelAsyncMock).toHaveBeenCalledWith(
  "anthropic", "claude-opus-4-6", undefined, undefined, {},
);

The useAsyncModelResolution: true path is supposed to skip the sync resolveModel and only invoke resolveModelAsync. The failure shows resolveModelMock is being invoked once with "anthropic", "claude-opus-4-6", suggesting either:

  • A separate code path inside prepareSimpleCompletionModel still falls through to resolveModel even when useAsyncModelResolution: true, OR
  • Mock state from a prior test in the file is leaking into this one (the file uses vi.hoisted for the mocks; if cleanup between tests isn't resetting call state, a previous prepareSimpleCompletionModel invocation could be the source of the recorded call)

The 1st-call payload shape ({authProfileId, preferredProfile} style) doesn't quite match the test's own arguments (it passes cfg: undefined, no profile fields), so the leaking-mock-state hypothesis is the more likely one — but I haven't tried to reproduce locally.

Impact

Currently affecting at least PR #90579; almost certainly affecting other PRs that have rebased onto / merged in main since 2026-06-11T02:12Z. Two CI lanes red on every affected PR until this is resolved.

Suggested follow-up

A short repro / fix from @vincentkoc would be quickest since the test was added in the same commit as the production change. If it's mock-state leakage, an explicit vi.clearAllMocks() in a beforeEach (or per-test reset of just the hoisted mocks) is usually enough. If it's a genuine code-path issue, the assertion expect(hoisted.resolveModelMock).not.toHaveBeenCalled() is correctly catching it and the fix is in prepareSimpleCompletionModel.

cc @steipete (PR review queue context)

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions