Skip to content

fix(core): keep estimated token split summing to total#5420

Merged
wenshao merged 1 commit into
QwenLM:mainfrom
he-yufeng:fix/usage-token-estimate-sum
Jun 19, 2026
Merged

fix(core): keep estimated token split summing to total#5420
wenshao merged 1 commit into
QwenLM:mainfrom
he-yufeng:fix/usage-token-estimate-sum

Conversation

@he-yufeng

Copy link
Copy Markdown
Contributor

What

When an OpenAI-compatible response reports only total_tokens (no prompt/completion breakdown), the converter estimates a 70/30 split. It rounded each half on its own:

finalPromptTokens = Math.round(totalTokens * 0.7);
finalCompletionTokens = Math.round(totalTokens * 0.3);

so the two halves could add up to more than the real total — e.g. total = 5 gives 4 + 2 = 6. This derives the completion half from the remainder instead, in both the non-streaming (convertOpenAIResponseToGemini) and streaming (convertOpenAIChunkToGemini) paths.

Why

usageMetadata exposes promptTokenCount, candidatesTokenCount and totalTokenCount. With the old rounding, promptTokenCount + candidatesTokenCount could exceed totalTokenCount, so anything that reconciles the split against the total (billing, logging, telemetry) sees inconsistent numbers. totalTokens is the one real figure we have here, so the split should always add back up to it.

Reviewer Test Plan

npx vitest run packages/core/src/core/openaiContentGenerator/converter.test.ts — see keeps the estimated prompt/completion split summing to total tokens, which feeds { prompt_tokens: 0, completion_tokens: 0, total_tokens: 5 } and asserts the split sums to 5. It fails before the change (expected 6 to be 5) and passes after.

Risk

Low. Only the total_tokens-only estimate path changes; the prompt half keeps the same Math.round(total * 0.7) value and the completion half becomes total - prompt. Cases that already report a real breakdown are untouched.

@wenshao

wenshao commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

@qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Thanks for the PR!

Template looks good ✓ — headings present, reviewer test plan with a concrete failing-then-passing assertion.

On direction: this is a clean, obvious bug fix. Math.round on each half independently doesn't preserve the invariant that the parts sum to the total — classic rounding error. Token accounting (billing, logging, telemetry) needs that invariant, so this is worth fixing. No direct Claude Code CHANGELOG reference, but the area (usage/token tracking) is clearly core.

On approach: the fix is exactly the minimal change — one line per path (non-streaming + streaming), deriving the completion half from the remainder. No scope creep, no unrelated edits. The test directly asserts the invariant that was broken. Nothing to cut.

Moving on to code review. 🔍

中文说明

感谢贡献!

模板完整 ✓ — 各节齐全,reviewer test plan 包含了具体的"改动前失败、改动后通过"断言。

方向:这是一个明确的 bug 修复。对两半分别 Math.round 无法保证拆分之和等于总量——经典的舍入误差。token 统计(计费、日志、遥测)需要这个不变量,值得修复。Claude Code CHANGELOG 无直接引用,但该领域(用量/token 追踪)显然属于核心功能。

方案:改动最小化——非流式和流式路径各改一行,用总量减去 prompt 部分得到 completion 部分。无多余改动,无范围蔓延。测试直接断言了被破坏的不变量。无需裁剪。

进入代码审查 🔍

Qwen Code · qwen3.7-max

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Code Review

Independent proposal (written before reading the diff): the fix is obvious — compute one half with Math.round, derive the other from the remainder so the invariant holds. The PR does exactly this, applied consistently to both non-streaming (convertOpenAIResponseToGemini) and streaming (convertOpenAIChunkToGemini) paths. The approach matches perfectly.

No correctness bugs, no security issues, no regressions. Both paths are updated symmetrically. The comment is updated to explain the rationale. The test directly asserts the broken invariant. No AGENTS.md violations — minimal, focused, no abstractions, no scope creep.

Nothing to flag. ✅

Test Results

Before (main branch, without fix)

Running the new test against the original code:

cd packages/core && npx vitest run src/core/openaiContentGenerator/converter.test.ts \
  -t "keeps the estimated prompt/completion split"

 FAIL  src/core/openaiContentGenerator/converter.test.ts > OpenAIContentConverter > keeps the estimated prompt/completion split summing to total tokens

AssertionError: expected 6 to be 5

- Expected  5
+ Received  6

 ❯ src/core/openaiContentGenerator/converter.test.ts:2722:9
     2720|       expect(
     2721|         (usage?.promptTokenCount ?? 0) + (usage?.candidatesTokenCount …
     2722|       ).toBe(5);

 Test Files  1 failed (1)
      Tests  1 failed | 133 skipped (134)

Bug confirmed: Math.round(5 * 0.7) + Math.round(5 * 0.3) = 4 + 2 = 6, not 5.

After (this PR)

cd packages/core && npx vitest run src/core/openaiContentGenerator/converter.test.ts

 ✓ src/core/openaiContentGenerator/converter.test.ts (134 tests) 57ms

 Test Files  1 passed (1)
      Tests  134 passed (134)

All 134 tests pass, including the new invariant check. The fix correctly derives completionTokens = totalTokens - promptTokens so the split always sums to the total.

中文说明

代码审查

独立方案(读 diff 前写出):修复方法显而易见——用 Math.round 算一半,另一半用减法得出,保证不变量成立。PR 正是这样做的,非流式 (convertOpenAIResponseToGemini) 和流式 (convertOpenAIChunkToGemini) 路径对称应用。方案完全匹配。

无正确性 bug,无安全问题,无回归。两条路径对称更新,注释解释了原理,测试直接断言被破坏的不变量。无 AGENTS.md 违规——最小化、聚焦、无过度抽象、无范围蔓延。

无需标记的问题。✅

测试结果

改动前(main 分支,无修复):expected 6 to be 5 —— bug 确认:Math.round(5 * 0.7) + Math.round(5 * 0.3) = 4 + 2 = 6

改动后(本 PR):全部 134 个测试通过,包括新的不变量检查。修复正确地用 completionTokens = totalTokens - promptTokens 保证拆分之和等于总量。

Qwen Code · qwen3.7-max

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

This is a textbook bug fix — clean, minimal, and correct.

The rounding invariant (prompt + completion = total) was broken by independent Math.round calls on each half. The fix derives one half from the remainder, which is the standard technique. Both code paths (streaming and non-streaming) are updated symmetrically. The test directly asserts the invariant and fails convincingly before the fix (expected 6 to be 5) and passes after.

The diff is exactly what this bug needs: 2 production lines changed, 1 test added, nothing else. No abstractions, no drive-by refactors, no scope creep. If I had to maintain this in six months, I'd have nothing to complain about.

Approving. ✅

中文说明

这是一个教科书式的 bug 修复——干净、最小化、正确。

舍入不变量(prompt + completion = total)被对两半分别 Math.round 破坏了。修复方法是用减法得出另一半,这是处理此类不变量的标准技巧。两条代码路径(流式和非流式)对称更新。测试直接断言该不变量,改动前令人信服地失败(expected 6 to be 5),改动后通过。

diff 正好是这个 bug 所需的:改动 2 行生产代码,添加 1 个测试,无其他内容。无过度抽象,无顺手重构,无范围蔓延。六个月后维护这个代码,不会有任何可抱怨的。

批准。✅

Qwen Code · qwen3.7-max

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looks ready to ship. ✅

@wenshao

wenshao commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

✅ Local verification — recommend merge

Verified this on a clean worktree (real npm ci, full tsc build and esbuild bundle into the shipped dist/cli.js). Exercised the change at three levels: the author's unit test, an A/B on the compiled converter covering both code paths, and a real end-to-end run of the bundled CLI under tmux against a mock OpenAI‑compatible server. The fix is correct, minimal, and complete.

Scope confirmed

True diff vs origin/main: 38 / 4, 2 files — a 4‑line behavior change in convertOpenAIResponseToGemini (non‑streaming) and convertOpenAIChunkToGemini (streaming), plus one unit test. Completion is now derived as the remainder:

finalPromptTokens = Math.round(totalTokens * 0.7);
finalCompletionTokens = totalTokens - finalPromptTokens; // was Math.round(totalTokens * 0.3)

A repo‑wide grep confirms these are the only two sites using the 70/30 estimate, so the fix is complete. The if (totalTokens > 0 && promptTokens === 0 && completionTokens === 0) guard is unchanged, so providers that report a real prompt/completion breakdown are untouched (no regression).

1) Author's Reviewer Test Plan — reproduced exactly

  • Revert the 4‑line fix → the new test fails with expected 6 to be 5 ✔ (matches the PR description)
  • On the PR branch → full converter.test.ts suite 134/134 passing

2) A/B on the compiled converter — both paths

Imported the built OpenAIContentConverter from dist/ and fed {prompt_tokens:0, completion_tokens:0, total_tokens:N} for N ∈ {1,2,3,5,7,10,15,25,35,100,101} through both functions:

Build non‑streaming (...ResponseToGemini) streaming (...ChunkToGemini)
OLD (fix reverted) ❌ mismatch at N=5,15,25,35 — prompt+cand = total+1 ❌ same
NEW (PR) ✅ all consistent — prompt+cand == total ✅ all consistent

Example N=5: OLD → 4 + 2 = 6 ≠ 5; NEW → 4 + 1 = 5.
⚠️ The PR's added unit test only covers the non‑streaming function. The streaming convertOpenAIChunkToGemini is fixed too and is verified consistent here.

3) Real end-to-end under tmux

Drove the bundled qwen --output-format json (real HTTP, streaming — confirmed stream_options.include_usage in the mock's request log) against a mock server that reports only total_tokens: 5:

Build per API call session input + output vs total
OLD prompt=4, candidates=2, total=56 ≠ 5 8 + 4 = 12 ≠ 10 ❌ inconsistent
NEW prompt=4, candidates=1, total=55 == 5 8 + 2 = 10 == 10 ✅ consistent

The CLI also issues a background managed-auto-memory-extractor call, so the session aggregates two calls. Under the old code both calls overshoot and the inconsistency propagates into the session‑level usageMetadata (promptTokenCount + candidatesTokenCount > totalTokenCount) — exactly the billing/telemetry symptom the PR describes. The fix makes every call and the session reconcile.

Lint / build

eslint clean on both changed files; core tsc build passes.

Optional (non‑blocking)

Consider adding a sibling assertion for the streaming convertOpenAIChunkToGemini path in the unit test, since the fix touches it as well — behavior is already verified correct above.

🇨🇳 中文版(点击展开)

✅ 本地验证 — 建议合并

我在一个干净的 worktree 中完成了验证(真实 npm ci,完整 tsc 编译 以及 esbuild 打包出实际发布的 dist/cli.js)。从三个层面验证了本次改动:作者的单元测试、对编译产物转换器的 A/B(覆盖两条代码路径),以及在 tmux 下用打包后的真实 CLI 对接 mock OpenAI 兼容服务的端到端运行。修复正确、最小且完整。

改动范围确认

相对 origin/main 的真实 diff:38 / 4,2 个文件 —— 即 convertOpenAIResponseToGemini(非流式)与 convertOpenAIChunkToGemini(流式)中各 4 行的行为变更,外加一个单测。completion 现在按余数推导:

finalPromptTokens = Math.round(totalTokens * 0.7);
finalCompletionTokens = totalTokens - finalPromptTokens; // 原来是 Math.round(totalTokens * 0.3)

全仓 grep 确认整个代码库里使用 70/30 估算的仅有这两处,因此修复是完整的。if (totalTokens > 0 && promptTokens === 0 && completionTokens === 0) 这一守卫未改动,所以正常返回 prompt/completion 拆分的 provider 完全不受影响(无回归)。

1)复现作者的 Reviewer Test Plan

  • 把这 4 行修复回退 → 新测试以 expected 6 to be 5 失败 ✔(与 PR 描述一致)
  • 在 PR 分支上 → converter.test.ts 全量 134/134 通过

2)对编译产物转换器做 A/B —— 两条路径

dist/ 导入已编译OpenAIContentConverter,对 N ∈ {1,2,3,5,7,10,15,25,35,100,101}{prompt_tokens:0, completion_tokens:0, total_tokens:N} 分别跑两个函数:

构建 非流式(...ResponseToGemini 流式(...ChunkToGemini
(回退修复) ❌ N=5,15,25,35 处不一致 —— prompt+cand = total+1 ❌ 同上
(PR) ✅ 全部一致 —— prompt+cand == total ✅ 全部一致

例如 N=5:旧 → 4 + 2 = 6 ≠ 5;新 → 4 + 1 = 5
⚠️ PR 新增的单测只覆盖了非流式函数。流式的 convertOpenAIChunkToGemini 同样被修复,这里也已验证一致。

3)tmux 下的真实端到端

用打包后的 qwen --output-format json(真实 HTTP、流式 —— 已在 mock 的请求日志中确认 stream_options.include_usage)对接一个只返回 total_tokens: 5 的 mock 服务:

构建 单次 API 调用 会话 input + output vs total
prompt=4, candidates=2, total=56 ≠ 5 8 + 4 = 12 ≠ 10 ❌ 不一致
prompt=4, candidates=1, total=55 == 5 8 + 2 = 10 == 10 ✅ 一致

CLI 还会额外发起一个后台 managed-auto-memory-extractor 调用,所以会话层会聚合两次调用。在旧代码下两次调用都超出 total,不一致会一路传导到会话级的 usageMetadatapromptTokenCount + candidatesTokenCount > totalTokenCount)—— 正是 PR 所描述的计费/遥测症状。修复后每次调用与整个会话都能对齐。

Lint / 构建

两个改动文件 eslint 均无告警;core tsc 构建通过。

可选(不阻塞合并)

建议在单测里为流式 convertOpenAIChunkToGemini 路径补一条同样的断言,因为修复也涉及该函数 —— 其行为在上面已验证正确。

@wenshao
wenshao merged commit a55eae9 into QwenLM:main Jun 19, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants