Codex review: needs real behavior proof before merge. Reviewed July 16, 2026, 4:21 PM ET / 20:21 UTC.

Summary
The PR adds a shared priced-usage mapper, records usage and cost from response.incomplete in both OpenAI Responses stream processors, and adds synthetic-event regression coverage.

PR surface: Source +91, Tests +104. Total +195 across 7 files.

Reproducibility: no. for the linked report: there is no high-confidence current-main reproduction of user cancellation or first-byte timeout. The distinct completed-only handling of a server-generated response.incomplete event is source-reproducible and covered by constructed-event tests.

Review metrics: none identified.

Root-cause cluster
Relationship: partial_overlap
Canonical: #100954
Summary: The PR fixes a nearby completed-only terminal-accounting defect, but the canonical issue describes client-side cancellation and timeout behavior that has not been shown to emit response.incomplete.

Members:

• canonical: fix(llm): token usage mismatch during early stream aborts on Azure OpenAI models #100954 - It remains the canonical report for early client-abort usage drift and should stay open unless that path is separately reproduced and fixed.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🦐 gold shrimp
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

• Narrow the title/body and remove the closing claim for fix(llm): token usage mismatch during early stream aborts on Azure OpenAI models #100954 unless the actual cancellation or timeout path is fixed.
• [P1] Add a redacted real-provider before/after transcript for max_output_tokens, or obtain an explicit maintainer proof override after updating the PR body.
• After adding proof, update the PR body to trigger review; if it does not rerun, ask a maintainer to comment @clawsweeper re-review.

Proof guidance:

• [P1] Needs real behavior proof before merge: Both regressions inject a constructed response.incomplete event through the processors, but the PR contains no redacted after-fix output from a real OpenAI or Azure stream; add terminal output or logs with private endpoints, keys, account data, and deployment details removed. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

• [P2] Merging with the current closing reference could close fix(llm): token usage mismatch during early stream aborts on Azure OpenAI models #100954 while its reported user-cancellation and first-byte-timeout behavior remains unresolved.
• [P1] Without provider output, Azure or OpenAI terminal-event behavior and the resulting persisted cost cannot be confirmed outside the constructed stream harness.

Maintainer options:

1. Decide the mitigation before merge
   Treat this as a narrowly scoped server-incomplete accounting fix, keep fix(llm): token usage mismatch during early stream aborts on Azure OpenAI models #100954 open for the distinct client-abort behavior, and validate the narrowed change with a redacted real OpenAI or Azure truncation run before merge.
2. Pause or close
   Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

• [P1] The assigned reviewer should require scope correction and then decide whether live provider evidence is mandatory or a narrowly documented proof override is acceptable.

Maintainer decision needed

• Question: After the PR is narrowed so it no longer claims to fix client cancellation or first-byte timeout, should exact-event processor tests be accepted as sufficient proof for the server-generated response.incomplete accounting fix?
• Rationale: The code and tests support the narrower server-terminal behavior, but only a maintainer can approve an exception to the external-PR real-behavior proof gate when the contributor lacks provider access.
• Likely owner: vincentkoc — They are the assigned reviewer and the discussion has already directed this exact proof decision to them.
• Options:
  • Require narrow scope and live proof (recommended): Keep the linked issue open, correct the PR scope, and require a redacted OpenAI or Azure max_output_tokens run showing populated usage and cost.
  • Approve a narrow proof override: Keep the linked issue open, correct the PR scope, and explicitly accept the upstream SDK contract plus both exact-event processor regressions instead of a live provider run.
  • Pause pending provider access: Leave the PR open until someone with an OpenAI or Azure deployment can capture the requested runtime evidence.

Security
Cleared: The diff changes internal usage mapping and tests without adding dependencies, external code execution, credentials handling, workflow permissions, or supply-chain surface.

Review findings

• [P1] Separate server-incomplete handling from client aborts — packages/ai/src/providers/openai-responses-shared.test.ts:2075

@clawsweeper re-review

The prior review was against an earlier commit. Since then this PR:

• Computes cost for response.incomplete usage (the [P2] finding) by routing it through the same terminal usage+cost path as response.completed via a shared recordResponsesTerminalUsage helper, with a regression assertion that incomplete usage produces nonzero cost.
• Applies the same fix to the sibling processor in src/agents/openai-transport-stream.ts, which had the identical completed-only handling ([P1] risk raised in the last review).

The PR body has been updated with the reasoning and evidence.

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Thanks for taking this @vincentkoc.

ClawSweeper re-reviewed at head (56d48ba) and split the verdict: patch quality 🐚 platinum hermit, proof 🧂 unranked krab. Overall follows the weaker of the two, so the only thing blocking merge is a redacted live transcript of a Responses stream that ends in response.incomplete.

I don't have access to an OpenAI or Azure provider key, so I can't capture that transcript myself. Rather than leave it stalled, here's the exact repro if anyone on the team wants to run it — it's a single call and costs about a cent:

• Set maxTokens low (e.g. 16) on any OpenAI/Azure Responses model. It lands in params.max_output_tokens (openai-responses-shared.ts:522), the server truncates the output, and the stream terminates with response.incomplete carrying incomplete_details.reason: "max_output_tokens" — response.completed never arrives.
• Run it once on main: reported token usage and cost come back zero.
• Run the same prompt on this branch: the token buckets and usage.cost.total are populated.

That before/after pair is precisely the behavior the review asks for, and it also closes the caveat in the review itself ("I did not establish a live Azure/OpenAI early-abort reproduction"). Endpoint, key and deployment name would need redacting before posting.

If capturing that isn't worth a maintainer's time, the alternative is to merge on patch quality with human judgment — the diff is confined to two terminal stream branches, CI is green, and security was cleared. Happy either way; just let me know which and I'll rebase (the branch is currently behind main).

Rebased onto current main (new head a0655cfc95c) to clear the merge conflict.

The conflict came from the module splits that landed since (#105894, #106006, #106354): src/agents/openai-transport-stream.ts is now a facade, so the sibling fix was ported to its new home — recordResponsesTerminalUsage + the response.incomplete branch live in src/agents/openai-responses-transport.ts, and the regression test moved into src/agents/openai-transport-stream.base.test-utils.ts (following the makeResponsesModel idiom used there). The packages/ai side rebased cleanly with no manual changes. Behavior is identical to the previously reviewed 56d48ba.

Local verification: both affected suites pass (openai-transport-stream.test.ts + openai-responses-shared.test.ts, 714 tests), typecheck clean on the touched files.

CI status after the rebase: 65 checks green, including checks-fast-loc-ratchet (the inline terminal-usage helpers moved to dedicated small modules — src/agents/openai-responses-terminal-usage.ts and packages/ai/src/providers/openai-responses-terminal-usage.ts — so both legacy files now sit below their pre-change line counts; no behavior change).

The one remaining red, check-dependencies, is not introduced by this PR: the same dead-export finding (attempt-session.ts: createEmbeddedAgentSessionWithResourceLoader) fails on current main HEAD (345083a) as well — see its check-dependencies run. Happy to rebase again once that's fixed upstream if a clean board is needed for merge.

Rebased onto current main (new head c22256d9a90), no conflicts and no code changes.

The check-dependencies failure reported above was inherited from the previous base snapshot — the dead-export it flagged (attempt-session.ts: createEmbeddedAgentSessionWithResourceLoader) has since been removed on main, so this rebase should bring CI to fully green. Locally re-verified after the rebase: 714/714 tests across both stream-processor suites, LOC ratchet clean, no type errors in the touched files.

CI is now fully green at head b41c88d98f7 — 68 checks passing, zero failures, including check-dependencies.

For the record: the brief red burst earlier today was inherited from main (the mergeChildEnv regression from #103918, fixed upstream in #107258 within ~30 minutes); none of the affected files are touched by this PR. Rebased once more after the upstream fix landed.

Nothing outstanding on my side — the proof-question framed in the review above remains the only open item.

Addressed the [P1] "use one canonical terminal-usage helper" finding, and rebased onto current main (new head ac75a646197).

The billing invariant — priced bucket split, calculateCost, and service-tier pricing — now lives in a single exported mapResponsesTerminalUsage in packages/ai/src/providers/openai-responses-terminal-usage.ts, re-exported via @openclaw/ai/internal/openai. Both Responses processors call it, so there is no longer a second implementation to drift.

On the one field that stays agent-local: the agent transport keeps a thin adapter that adds only reasoningTokens. That is deliberate — the normalized Usage type (packages/llm-core/src/types.ts) does not model reasoningTokens (it is tracked separately by the agent usage accumulator and diagnostics), so the shared mapper returns a Usage and the agent layers reasoning on top. The adapter contains no billing logic, and the previous as never casts are gone.

Local verification after the rebase:

• openai-responses-shared.test.ts → 52 passed
• openai-transport-stream.test.ts → 331 passed
• usage.normalization.test.ts → 10 passed
• typecheck clean on the touched files; LOC ratchet clean.

The remaining open item is unchanged: the proof-override vs. live-transcript decision for the response.incomplete path.

@clawsweeper re-review

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

• State: Complete
• Detail: The targeted re-review published an exact durable review comment and queued an executing target-wide serialized router scan.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/29349571627
• Updated: 2026-07-14T16:38:27.125Z

CI note: the red checks-node-compact-small-1 (→ ci-gate) is not introduced by this PR — it is inherited from main.

The failing assertion is test/scripts/control-ui-i18n.test.ts → control-ui catalog drift: orphan keys configView.unsavedChange/configView.*, quickSettings.pending.*, and mcpPage.publishing in every locale catalog. These keys were removed from the English source (ui/src/i18n/locales/en.ts) by #107458 ("auto-save config edits… single restart affordance") without a matching locale-catalog regen, so scripts/control-ui-i18n-verify.ts flags the locales as orphaned.

Evidence it is a base issue, not mine:

• This PR's diff touches only the three Responses terminal-usage billing files — no ui/, no i18n.
• The i18n catalog and source files are byte-identical between this branch's base and current main HEAD (c8e94b202e0); the key-removing commit improve(ui): auto-save config edits, flatten config pages, single restart affordance #107458 is present in both.
• Running control-ui-i18n-verify verify reproduces the exact orphan list on a clean checkout of current main.

Nothing to fix on this branch — hand-editing the locale catalogs would be out of scope for a billing change and would collide with main's own upcoming locale refresh (cf. #107372). I'll rebase once main lands the control-ui locale regen so the board goes green again. The billing consolidation itself is verified locally: openai-responses-shared.test.ts (52), openai-transport-stream.test.ts (331), usage.normalization.test.ts (10) pass; typecheck and LOC ratchet clean.

@vincentkoc — the re-review at ac75a64 closed the last code-side item: the [P1] "use one canonical terminal-usage helper" finding is confirmed resolved, and patch quality is back to 🦞 diamond lobster with no remaining code findings. Per ClawSweeper's own "maintainer decision needed" note, the only gate left is the proof question — and it's addressed to you as the assigned reviewer.

To keep the call low-friction, the two options are unchanged:

• Require live proof — I capture a redacted before/after response.incomplete transcript showing populated usage + nonzero cost. I don't have an OpenAI/Azure key, so this needs someone with provider access. The repro is a single ~1¢ call: set maxTokens: 16 on any Responses model → the server truncates → the stream ends in response.incomplete with incomplete_details.reason: "max_output_tokens" and no response.completed.
• Narrow proof override — accept the official SDK contract plus the two exact-event processor regressions as sufficient for this provider-accounting fix.

Happy either way — just say which and I'll act.

On CI: the current reds are inherited from main, not from this diff. #107458 (config auto-save) landed ConfigPatchOptions.note and removed i18n keys without regenerating the locale catalogs, which breaks the ui/ type-check and i18n checks — none of which this billing change touches (the diff is confined to the three Responses terminal-usage files). I'll rebase to a green board as soon as main regenerates those catalogs.

CI note: the current red checks are not from this diff. Every failing test shard reports the same root cause:

Canonical SQLite schema contains non-STRICT tables: apns_registration_tombstones

That table was added to the canonical schema by #108543 (refactor(apns): move registrations to shared SQLite, merged to main at 08:52 UTC today) without the STRICT keyword (src/state/openclaw-state-schema.sql:640), which trips the schema-strictness gate for every test that opens the state DB (channels/ingress-queue suites, sqlite-session-flip proof, QA smoke, gateway-watch). This PR's diff touches only the Responses usage/billing files — no SQLite, no schema.

The remaining failure ("OpenGrep — PR Diff") is an infra error ("Failed to fetch available versions from GitHub" while downloading the scanner), unrelated to code.

I'll update the branch once main picks up the STRICT fix so this goes back to green (it was fully green at the previous round).

Follow-up on the CI note above: the schema fix landed on main as #108850 (91f57a31596, merged 09:41 UTC — after this branch's previous main merge at 08:55). Merged origin/main into the branch again (new head b61cd1edd22, no conflicts) so the merge-ref now includes the STRICT declaration. The PR's target suites still pass locally (65/65). CI should be back to green on this run.

Update: no maintainer re-run needed — both infra flakes cleared on their own.

I merged current main (a9f8283) into the branch. That push started a fresh CI round, and both tool-install flakes passed on clean runners. Head ff7e970c78e is now fully green: 86 passing, 0 failing.

On the "Merge conflicts" banner that showed up earlier: there was no real conflict. GitHub's refs/pull/101608/merge ref was still based on a main commit 239 commits back, so the mergeability state was stale. Merging current main produced zero conflicts and the PR reads as mergeable again.

No code changes in this update — the diff is still the same usage mapping plus its regression tests, and the targeted suites pass locally (65/65).

_{Original note, kept for context: the two reds were Install opengrep (Failed to fetch available versions from GitHub) and Install iOS Swift tooling (/opt/homebrew/Cellar/xcodegen/2.46.0 is not a directory); both died before any scan ran, and this PR touches no iOS/Swift code.}

Closing this in favor of #109904, which fixes what is left of #100954 in a much narrower change.

Why: #109615 (merged) rebuilt the package-side processor to finalize response.completed and
response.incomplete through one finalizeResponse, which records usage, cost and service-tier pricing.
That is the same fix this PR was making in packages/ai/src/providers/openai-responses-shared.ts, so that
half is now redundant — and it is where the branch conflicts against current main.

What #109615 did not touch is the agent-side processor. On current main,
src/agents/openai-responses-transport.ts:1701 still has a terminal branch for response.completed
only, so response.incomplete matches nothing in the chain and is dropped: no usage, no stop reason.
That path is live for the openclaw-openai-responses-transport and
openclaw-azure-openai-responses-transport runtimes, so #100954's Azure drift is still reproducible
today. #109904 fixes that surface alone, with the added tests failing on main with
expected +0 to deeply equal 50.

Rather than rebase 342 commits of drift and then delete half the diff, #109904 is a fresh, smaller change
against current main.

One note that may be useful for the status: 📣 needs proof gate here, and thanks for the patience on it:
#109615 pinned the identical package-side behavior with a synthetic terminal event
(it("finalizes incomplete terminal events with usage"),
packages/ai/src/providers/openai-responses-shared.test.ts:1281) rather than a captured provider
transcript. #109904 uses that same proof shape against the agent processor, so it should not need a live
capture either — but happy to follow whatever you prefer there.

@vincentkoc unassigning yourself is fine — no action needed on this one.