fix(core): keep token escalation warm across agent rounds#5062
Conversation
| config: { | ||
| abortSignal: roundAbortController.signal, | ||
| tools: [{ functionDeclarations: toolsList }], | ||
| ...(stickyMaxOutputTokens !== undefined |
There was a problem hiding this comment.
[Suggestion] Redundant re-escalation on subsequent truncated rounds
hasUserMaxTokensOverride in geminiChat.ts (line 1889) only checks cgConfig.samplingParams.max_tokens and QWEN_CODE_MAX_OUTPUT_TOKENS — it does not see params.config.maxOutputTokens. When this sticky value flows through to a subsequent agent round and the model still hits MAX_TOKENS at the already-escalated limit, GeminiChat's inner escalation block fires again: maxTokensEscalated resets per call, hasUserMaxTokensOverride stays false, so all guards pass. The escalation computes Math.max(ESCALATED_MAX_TOKENS, tokenLimit(model, 'output')) — the identical value — pops the partial response from history, yields a spurious RETRY, and makes a duplicate API call at the same limit.
This wastes one API call (with full request history) per subsequent truncated round. In a long agent session with many truncation-heavy rounds, the cost compounds.
Fix options:
- Broaden
hasUserMaxTokensOverrideto also checkparams.config?.maxOutputTokens !== undefined, or - Add a guard in the escalation condition:
&& (params.config?.maxOutputTokens ?? 0) < escalatedLimit
| ...(stickyMaxOutputTokens !== undefined | |
| ...(stickyMaxOutputTokens !== undefined | |
| ? { maxOutputTokens: stickyMaxOutputTokens } | |
| : {}), |
— qwen3.7-max via Qwen Code /review
| } | ||
| // Signal UI to discard partial output | ||
| yield { type: StreamEventType.RETRY }; | ||
| yield { |
There was a problem hiding this comment.
[Suggestion] Misleading log message when sticky escalation is active
The log at line 2219 reads "Output truncated at capped default. Escalating to ${escalatedLimit} tokens." — but when stickyMaxOutputTokens is already applied via params.config.maxOutputTokens, the request was not at the capped default (8K); it was already at the escalated limit (e.g., 65536). An oncall engineer reading this log would conclude the request used 8K when it actually used 65536, leading to misdiagnosis.
Consider including the actual starting limit:
debugLogger.info(
`Output truncated at ${params.config?.maxOutputTokens ?? 'capped default'}. Escalating to ${escalatedLimit} tokens.`,
);Or skip the log + escalation entirely when no real escalation would occur (i.e., params.config.maxOutputTokens >= escalatedLimit).
— qwen3.7-max via Qwen Code /review
|
@qwen-code /triage |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Hi @he-yufeng — thanks for the PR! The fix itself looks well-scoped, but the PR body doesn't follow our PR template. A few required sections are missing:
- What this PR does / Why it's needed — the
## Summarysection covers the "what" but the template asks for these as separate headings so reviewers can quickly distinguish the change from the motivation. - Reviewer Test Plan —
## To verifylists commands, but the template needs the full structure:### How to verify(reproduction steps),### Evidence (Before & After)(output or N/A for non-UI changes), and the### Tested onOS table. - Risk & Scope — tradeoffs, out-of-scope items, breaking changes.
- Linked Issues — formal
Fixes #4964so the issue auto-closes on merge. - Chinese translation — the
<details>block with a full translation of the English body.
Could you update the PR description to match the template? Happy to re-run triage once that's done.
中文说明
你好 @he-yufeng — 感谢你的 PR!修复本身范围合理,但 PR 描述没有按照我们的 PR 模板 填写,缺少以下必要部分:
- What this PR does / Why it's needed —
## Summary涵盖了"做了什么",但模板要求分成两个独立标题,方便审阅者快速区分变更内容和动机。 - Reviewer Test Plan —
## To verify列出了命令,但模板需要完整结构:### How to verify(复现步骤)、### Evidence (Before & After)(输出或非 UI 变更写 N/A)、以及### Tested on操作系统表格。 - Risk & Scope — 权衡取舍、不在范围内的内容、破坏性变更。
- Linked Issues — 正式的
Fixes #4964,以便合并后自动关闭 issue。 - 中文说明 —
<details>块内包含英文正文的完整翻译。
请按照模板更新 PR 描述,完成后可以重新触发审查。
— Qwen Code · qwen3.7-max
Signed-off-by: Yufeng He <[email protected]>
a39064b to
cd8d4f1
Compare
|
Addressed the requested changes in the latest force-push.\n\nWhat changed:\n- GeminiChat now checks the per-request maxOutputTokens before automatic escalation. If the request already uses the escalated limit, it does not emit another identical escalation retry or make a duplicate stream call.\n- The escalation log now reports the actual starting limit when one is present, instead of always saying capped default.\n- Added a core regression test for the sticky-limit case, alongside the existing agent-headless test that verifies the escalated limit is carried into the next agent round.\n- Updated the PR body to the repository template.\n\nValidated on Windows:\n- |
wenshao
left a comment
There was a problem hiding this comment.
LGTM ✅ — The sticky escalation mechanism is clean and well-scoped. Both regression tests validate the core behavior correctly. The two prior inline suggestions (redundant re-escalation, misleading log) are addressed by the shouldEscalateMaxOutputTokens guard and the improved startingLimitLabel in this commit.\n\n_— qwen3.7-max via Qwen Code /review_
Local verification (merge reference)I built Verdict: works as designed and reproduces/fixes #4964 on the wire. The escalated output budget is now preserved into the next agent round after a tool call; the re-escalation guard and the warm-escalation logic are both correct. LGTM. 1. Live A/B in the real binary — the headlineSame fork scenario on the pre-fix (merge-base) vs PR binary. The mock records the
The only behavioral difference is round 2's budget: pre-fix drops back to the 8K cap (so the next response would truncate again — exactly #4964), and the PR keeps the 64K escalated limit warm. Verified end-to-end through the real 2. GeminiChat escalation also confirmed liveA separate headless 3. PR test plan — green
4. Mutation — both new tests are non-vacuousReverting So the sticky-carry and the re-escalation guard are each genuinely guarded. Notes
Good to merge. The behavior is correct, reproduced and fixed on the real wire via the 中文版(Chinese version)本地验证(合并参考)我构建了 结论:行为符合设计,并在"线缆"上复现并修复了 #4964。升级后的 output 预算现在会保留到工具调用后的下一轮 agent;re-escalation 守卫与"保温升级"逻辑均正确。建议合并。 1. 真实二进制 A/B —— 要点在修复前(merge-base)与 PR 二进制上跑同一个 fork 场景。mock 记录 fork 的
唯一的行为差异在第 2 轮的预算:修复前回退到 8K 上限(下一次响应会再次截断——正是 #4964),PR 则让 64K 升级 limit 保持"温热"。这是通过真实 2. GeminiChat 升级也已线上确认另跑一个 headless 3. PR 测试计划 —— 全绿
4. 变异测试 —— 两个新测试都非空壳把 所以 sticky 保留与 re-escalation 守卫都被真正守护。 说明
可以合并。 行为正确,已在真实 |
|
@qwen-code /triage |
|
Thanks for the PR! Template looks good ✓ — all required headings present, bilingual, test plan clear. On direction: This is a clean bug fix for #4964 — headless agent rounds lose the escalated output-token budget after a tool call, causing repeated truncation. The CHANGELOG has recent related work ( On approach: The scope is tight — 150 additions across 4 files, two changes that map 1:1 to the two failure modes:
This is about as minimal as it gets. The 中文说明感谢贡献! 模板完整 ✓ — 所有必需章节齐全,双语,测试计划清晰。 方向: 这是对 #4964 的干净 bug 修复 — headless agent 在工具调用后丢失升级后的 output-token 预算,导致反复截断。CHANGELOG 中有近期相关工作( 方案: 范围紧凑 — 4 个文件 150 行新增,两处改动一一对应两种失败模式:
这基本上是最小化方案。 — Qwen Code · qwen3.7-max |
Code ReviewIndependent proposal matches the PR: track escalated limit across agent rounds via a sticky variable, and guard against duplicate escalation in GeminiChat. The diff implements exactly that. No blockers. The logic is clean:
Tests are non-vacuous: the new agent-headless test verifies round-2 gets the escalated limit, the new geminiChat test verifies no duplicate escalation when already at the limit. CI mutation (by maintainer) confirmed both tests fail when the fix is reverted. Test Results
Real-Scenario Test (tmux)The bug requires a provider that returns
CLI smoke test output (headless No errors, no regression. 中文说明代码审查独立方案与 PR 一致:通过 sticky 变量跨 agent 轮次追踪升级后的 limit,并在 GeminiChat 中防止重复升级。diff 精确实现了这一点。 无阻塞问题。 逻辑清晰:
测试非空壳:新 agent-headless 测试验证第 2 轮获得升级后的 limit,新 geminiChat 测试验证已处于升级 limit 时不重复升级。CI 变异测试(由维护者执行)确认两个新测试在回退修复后均失败。 测试结果
真实场景测试 (tmux)该 bug 需要返回 — Qwen Code · qwen3.7-max |
ReflectionThis is a textbook bug fix. The problem is real and well-documented (#4964), the fix is minimal (150 lines, 2 source files), and the approach is the obvious one — carry the escalated limit forward, don't re-escalate when already there. Looking back at my independent proposal from Stage 2: the PR matches it exactly. I can't find a simpler path. The two changes (sticky carry in AgentCore, re-escalation guard in GeminiChat) each address one failure mode, and neither adds complexity that doesn't directly solve the problem. The tests are honest — both new tests fail when the fix is reverted, and the 201-test suite is clean. CI is green across all three platforms. The maintainer's live wire-level A/B testing (documented in a comment on this PR) provides the definitive proof that the fix works end-to-end. Risk is low: the sticky limit is scoped to AgentCore's reasoning loop, so the main conversation loop is unaffected. The only tradeoff — keeping a truncated partial response instead of making an identical duplicate API call — is clearly the right choice. Verdict: Approve. Clean fix, real bug, well-tested, ready to ship. ✅ 中文说明总结这是一个教科书级的 bug 修复。问题真实且有据可查(#4964),修复最小化(150 行,2 个源文件),方案直观——将升级后的 limit 向后传递,已处于升级状态时不再重复升级。 回顾我在 Stage 2 的独立方案:PR 与之完全一致。找不到更简路径。两处改动(AgentCore 中的 sticky 传递、GeminiChat 中的再升级守卫)各解决一种失败模式,且都没有增加与问题无关的复杂度。 测试诚实——两个新测试在回退修复后均失败,201 个测试的套件全部通过。CI 在三个平台上全绿。维护者的线上 A/B 测试(记录在本 PR 评论中)提供了修复端到端生效的决定性证据。 风险低:sticky limit 作用域仅限 AgentCore 的推理循环,主对话循环不受影响。唯一的取舍——保留截断的 partial response 而非发出相同的重复 API 调用——显然是正确的选择。 结论:批准。 干净的修复,真实的 bug,测试充分,可以合并。✅ — Qwen Code · qwen3.7-max |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
Signed-off-by: Yufeng He <[email protected]>
What this PR does
This PR keeps automatic output-token escalation warm across the rest of a headless agent run. When
GeminiChathas to retry a truncated response with a largermaxOutputTokens,AgentCorenow carries that escalated limit into later tool-result rounds instead of dropping back to the capped default. It also preventsGeminiChatfrom re-escalating when the current request is already using the escalated output limit.Why it's needed
In #4964, a headless agent could successfully retry a truncated first response, execute a tool call, and then truncate again on the next agent round because the larger output budget was not preserved. The first version of this PR preserved the budget at the agent layer, but review caught one remaining edge:
GeminiChatstill treated the per-request sticky limit as non-user config, so a laterMAX_TOKENSfinish could emit another identical escalation retry and duplicate API call. This update closes that edge without changing user-provided or environment-provided token limits.Reviewer Test Plan
How to verify
Run the core tests below. The new regression checks that a request already carrying the escalated
maxOutputTokensdoes not emit anothermaxOutputTokensEscalatedretry and does not make a duplicate stream call. The existing agent-headless regression checks that the escalated limit is preserved into the next agent round after a tool call.Evidence (Before & After)
N/A. This is non-UI agent/core behavior.
Tested on
Environment (optional)
Windows 11, Node/npm from the repo workspace.
Validated locally:
One parallel verification attempt ran two coverage-enabled Vitest targets at the same time and hit a
coverage/.tmp/coverage-0.jsonENOENT while the tests themselves had passed. Re-running the affected target by itself passed.Risk & Scope
Linked Issues
Fixes #4964
中文说明
这个 PR 做了什么
这个 PR 让 headless agent run 中的自动 output-token 升级在后续轮次继续生效。
GeminiChat因为响应被截断而用更大的maxOutputTokens重试后,AgentCore会把这个升级后的限制带到后面的工具结果轮次里,而不是回到 capped default。同时,如果当前请求已经使用升级后的 output limit,GeminiChat不会再触发一次相同额度的升级重试。为什么需要
在 #4964 里,headless agent 可能第一轮响应被截断,重试成功并执行工具调用;但下一轮 agent 响应又回到较小输出预算,导致再次截断。这个 PR 的第一版已经在 agent 层保留了预算,但 review 指出还有一个边界:
GeminiChat没有把 per-request sticky limit 视为已经升级过的限制,所以后续MAX_TOKENS仍可能发出一次相同额度的 retry,造成重复 API 调用。本次更新补上这个边界,同时不改变用户显式配置或环境变量里的 token limit。Reviewer Test Plan
How to verify
运行上面的 core 测试。新增回归测试会确认:当请求已经携带升级后的
maxOutputTokens时,不会再发出maxOutputTokensEscalatedretry,也不会重复调用 stream API。已有 agent-headless 回归测试会确认:工具调用后的下一轮 agent round 仍然复用升级后的限制。Evidence (Before & After)
N/A。这是非 UI 的 agent/core 行为。
Tested on
同英文表格。
Environment (optional)
Windows 11,本地仓库 Node/npm 环境。
Risk & Scope
Linked Issues
Fixes #4964