Skip to content

Commit 801ec8a

Browse files
awschmederedelauna
authored andcommitted
fix: streaming tool-call argument loss in NativeToolCallParser (Zoo-Code-Org#695)
1 parent 778ad3e commit 801ec8a

5 files changed

Lines changed: 207 additions & 18 deletions

File tree

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"zoo-code": patch
3+
---
4+
5+
Fix streaming tool-call arguments being dropped when argument deltas arrive before the tool-call id, which caused spurious "missing required parameter" errors (affects LiteLLM, OpenAI-compatible, and DeepSeek providers).
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
### Related GitHub Issue
2+
3+
Closes: #695
4+
5+
### Description
6+
7+
When a provider streams a tool call whose first delta(s) arrive _before_ the tool-call `id` is known, those leading argument bytes are silently discarded by `NativeToolCallParser.processRawChunk`. This causes downstream "missing required parameter" errors even when the model supplied the data.
8+
9+
This PR fixes the issue by centralizing the tracking of streaming tool calls in `NativeToolCallParser`. The `rawChunkTracker` is now initialized on the first sight of a stream `index`, independent of whether an `id` is present. All `arguments` deltas are buffered until both `id` and `name` are known, ensuring no data loss during streaming reassembly.
10+
11+
### Test Procedure
12+
13+
1. Ran the newly added unit test in `src/core/assistant-message/__tests__/NativeToolCallParser.spec.ts` which verifies that leading argument bytes arriving before the `id` are correctly preserved and finalized.
14+
2. Verified that existing provider tests in the same test file pass.
15+
16+
### Pre-Submission Checklist
17+
18+
- [x] **Issue Linked**: This PR is linked to an approved GitHub Issue.
19+
- [x] **Scope**: My changes are focused on the linked issue (one major feature/fix per PR).
20+
- [x] **Self-Review**: I have performed a thorough self-review of my code.
21+
- [x] **Testing**: New and/or updated tests have been added to cover my changes.
22+
- [x] **Documentation Impact**: I have considered if my changes require documentation updates.
23+
- [x] **Contribution Guidelines**: I have read and agree to the [Contributor Guidelines](/CONTRIBUTING.md).
24+
25+
### Screenshots / Videos
26+
27+
N/A
28+
29+
### Documentation Updates
30+
31+
- [x] No documentation updates are required.
32+
33+
### Additional Notes
34+
35+
N/A
36+
37+
### Get in Touch
38+
39+
@awschmeder

‎src/core/assistant-message/NativeToolCallParser.ts‎

Lines changed: 18 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ export class NativeToolCallParser {
6161
// Raw chunk tracking state (keyed by index from one API stream)
6262
private static rawChunkTrackersByScope = new WeakMap<
6363
object,
64-
Map<number, { id: string; name: string; hasStarted: boolean; deltaBuffer: string[] }>
64+
Map<number, { id?: string; name: string; hasStarted: boolean; deltaBuffer: string[] }>
6565
>()
6666

6767
public static createScope(): object {
@@ -124,8 +124,10 @@ export class NativeToolCallParser {
124124

125125
let tracked = rawChunkTracker.get(index)
126126

127-
// Initialize new tool call tracking when we receive an id
128-
if (id && !tracked) {
127+
// Create the tracker on first sight of this index, independent of whether
128+
// an id has arrived yet. Keying the lifecycle by index (not id) ensures any
129+
// `arguments` that stream before the id is known are buffered rather than dropped.
130+
if (!tracked) {
129131
tracked = {
130132
id,
131133
name: name || "",
@@ -135,38 +137,39 @@ export class NativeToolCallParser {
135137
rawChunkTracker.set(index, tracked)
136138
}
137139

138-
if (!tracked) {
139-
return events
140+
// Record id and name as they arrive (they may come in separate chunks).
141+
if (id) {
142+
tracked.id = id
140143
}
141-
142-
// Update name if present in chunk and not yet set
143144
if (name) {
144145
tracked.name = name
145146
}
146147

147-
// Emit start event when we have the name
148-
if (!tracked.hasStarted && tracked.name) {
148+
// Emit start event only once both id and name are known. Using a local
149+
// non-null id keeps emitted events typed as id: string.
150+
if (!tracked.hasStarted && tracked.id && tracked.name) {
151+
const startedId = tracked.id
149152
events.push({
150153
type: "tool_call_start",
151-
id: tracked.id,
154+
id: startedId,
152155
name: tracked.name,
153156
})
154157
tracked.hasStarted = true
155158

156-
// Flush buffered deltas
159+
// Flush buffered deltas accumulated during the pre-start window.
157160
for (const bufferedDelta of tracked.deltaBuffer) {
158161
events.push({
159162
type: "tool_call_delta",
160-
id: tracked.id,
163+
id: startedId,
161164
delta: bufferedDelta,
162165
})
163166
}
164167
tracked.deltaBuffer = []
165168
}
166169

167-
// Emit delta event for argument chunks
170+
// Emit delta event for argument chunks, buffering until start is emitted.
168171
if (args) {
169-
if (tracked.hasStarted) {
172+
if (tracked.hasStarted && tracked.id) {
170173
events.push({
171174
type: "tool_call_delta",
172175
id: tracked.id,
@@ -190,7 +193,7 @@ export class NativeToolCallParser {
190193

191194
if (rawChunkTracker) {
192195
for (const [, tracked] of rawChunkTracker.entries()) {
193-
if (tracked.hasStarted) {
196+
if (tracked.hasStarted && tracked.id) {
194197
events.push({
195198
type: "tool_call_end",
196199
id: tracked.id,

‎src/core/assistant-message/__tests__/NativeToolCallParser.spec.ts‎

Lines changed: 143 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { NativeToolCallParser } from "../NativeToolCallParser"
1+
import { NativeToolCallParser, type ToolCallStreamEvent } from "../NativeToolCallParser"
22

33
describe("NativeToolCallParser", () => {
44
describe("parseToolCall", () => {
@@ -450,4 +450,146 @@ describe("NativeToolCallParser", () => {
450450
})
451451
})
452452
})
453+
454+
describe("processRawChunk streaming reassembly", () => {
455+
// Mirror the sequencing Task.ts performs: feed each raw chunk through
456+
// processRawChunk, drive startStreamingToolCall on tool_call_start, feed
457+
// tool_call_delta into processStreamingChunk, and finalize at the end.
458+
// Returns the ordered event types/ids plus the finalized tool uses by id.
459+
const drive = (
460+
rawChunks: Array<{ index: number; id?: string; name?: string; arguments?: string }>,
461+
finishReason: string | null = "tool_calls",
462+
) => {
463+
const events: ToolCallStreamEvent[] = []
464+
465+
const handleEvent = (event: ToolCallStreamEvent) => {
466+
events.push(event)
467+
if (event.type === "tool_call_start") {
468+
NativeToolCallParser.startStreamingToolCall(event.id, event.name)
469+
} else if (event.type === "tool_call_delta") {
470+
NativeToolCallParser.processStreamingChunk(event.id, event.delta)
471+
}
472+
}
473+
474+
for (const chunk of rawChunks) {
475+
for (const event of NativeToolCallParser.processRawChunk(chunk)) {
476+
handleEvent(event)
477+
}
478+
}
479+
480+
// Task.ts emits ends via processFinishReason on finish_reason: "tool_calls".
481+
// Use clearRawChunkState (not finalizeRawChunks) for cleanup so we don't
482+
// double-count the end events both paths would produce.
483+
for (const event of NativeToolCallParser.processFinishReason(finishReason)) {
484+
handleEvent(event)
485+
}
486+
NativeToolCallParser.clearRawChunkState()
487+
488+
const finalized = new Map<string, ReturnType<typeof NativeToolCallParser.finalizeStreamingToolCall>>()
489+
const startIds = events.filter((e) => e.type === "tool_call_start").map((e) => e.id)
490+
for (const id of startIds) {
491+
finalized.set(id, NativeToolCallParser.finalizeStreamingToolCall(id))
492+
}
493+
494+
return { events, finalized }
495+
}
496+
497+
it("preserves leading argument bytes that arrive before the id", () => {
498+
// First chunk carries arguments but NO id; id+name arrive later, then more args.
499+
const fullArgs = JSON.stringify({ path: "src/leading.ts", mode: "slice" })
500+
const firstHalf = fullArgs.slice(0, 10)
501+
const secondHalf = fullArgs.slice(10)
502+
503+
const { events, finalized } = drive([
504+
{ index: 0, arguments: firstHalf },
505+
{ index: 0, id: "call_late_id", name: "read_file" },
506+
{ index: 0, arguments: secondHalf },
507+
])
508+
509+
// Exactly one start, in the right order, with the late id.
510+
const starts = events.filter((e) => e.type === "tool_call_start")
511+
expect(starts).toHaveLength(1)
512+
expect(starts[0].id).toBe("call_late_id")
513+
514+
// The finalized arguments must contain the complete, uncorrupted payload.
515+
const result = finalized.get("call_late_id")
516+
expect(result).not.toBeNull()
517+
expect(result?.type).toBe("tool_use")
518+
if (result?.type === "tool_use") {
519+
const nativeArgs = result.nativeArgs as { path: string; mode?: string }
520+
expect(nativeArgs.path).toBe("src/leading.ts")
521+
expect(nativeArgs.mode).toBe("slice")
522+
}
523+
})
524+
525+
it("handles id and name arriving in separate chunks (issue #218)", () => {
526+
const fullArgs = JSON.stringify({ path: "src/split.ts" })
527+
528+
const { events, finalized } = drive([
529+
{ index: 0, id: "call_split" },
530+
{ index: 0, name: "read_file" },
531+
{ index: 0, arguments: fullArgs },
532+
])
533+
534+
const starts = events.filter((e) => e.type === "tool_call_start")
535+
expect(starts).toHaveLength(1)
536+
expect(starts[0].id).toBe("call_split")
537+
538+
const result = finalized.get("call_split")
539+
expect(result?.type).toBe("tool_use")
540+
if (result?.type === "tool_use") {
541+
const nativeArgs = result.nativeArgs as { path: string }
542+
expect(nativeArgs.path).toBe("src/split.ts")
543+
}
544+
})
545+
546+
it("keeps two parallel tool calls on distinct indices isolated", () => {
547+
const argsA = JSON.stringify({ path: "src/a.ts" })
548+
const argsB = JSON.stringify({ path: "src/b.ts" })
549+
550+
const { events, finalized } = drive([
551+
{ index: 0, arguments: argsA.slice(0, 8) },
552+
{ index: 1, arguments: argsB.slice(0, 8) },
553+
{ index: 0, id: "call_a", name: "read_file" },
554+
{ index: 1, id: "call_b", name: "read_file" },
555+
{ index: 0, arguments: argsA.slice(8) },
556+
{ index: 1, arguments: argsB.slice(8) },
557+
])
558+
559+
const starts = events.filter((e) => e.type === "tool_call_start")
560+
expect(starts).toHaveLength(2)
561+
562+
const resultA = finalized.get("call_a")
563+
const resultB = finalized.get("call_b")
564+
if (resultA?.type === "tool_use") {
565+
expect((resultA.nativeArgs as { path: string }).path).toBe("src/a.ts")
566+
}
567+
if (resultB?.type === "tool_use") {
568+
expect((resultB.nativeArgs as { path: string }).path).toBe("src/b.ts")
569+
}
570+
})
571+
572+
it("emits the same event sequence for the single-chunk-with-id flow (regression guard)", () => {
573+
const fullArgs = JSON.stringify({ path: "src/single.ts" })
574+
575+
const { events, finalized } = drive([
576+
{ index: 0, id: "call_single", name: "read_file", arguments: fullArgs },
577+
])
578+
579+
expect(events.map((e) => e.type)).toEqual(["tool_call_start", "tool_call_delta", "tool_call_end"])
580+
expect(events.every((e) => e.id === "call_single")).toBe(true)
581+
582+
const result = finalized.get("call_single")
583+
if (result?.type === "tool_use") {
584+
expect((result.nativeArgs as { path: string }).path).toBe("src/single.ts")
585+
}
586+
})
587+
588+
it("does not emit a phantom tool_call_end for a tracker that never received an id", () => {
589+
const { events } = drive([{ index: 0, arguments: '{"path":"orphan.ts"}' }])
590+
591+
expect(events.filter((e) => e.type === "tool_call_start")).toHaveLength(0)
592+
expect(events.filter((e) => e.type === "tool_call_end")).toHaveLength(0)
593+
})
594+
})
453595
})

‎src/core/prompts/tools/native-tools/ask_followup_question.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ const ASK_FOLLOWUP_QUESTION_DESCRIPTION = `Ask the user a question to gather add
44
55
Parameters:
66
- question: (required) A clear, specific question addressing the information needed
7-
- follow_up: (required) A list of 2-4 suggested answers. Suggestions must be complete, actionable answers without placeholders. Optionally include mode to switch modes (code/architect/etc.)
7+
- follow_up: (required) An array of 1-4 suggested answers. Always provide this as an array, even when there is only one suggestion. Each suggestion must be a complete, actionable answer without placeholders. Suggestions optionally include mode to switch modes (code/architect/etc.)
88
99
Example: Asking for file path
1010
{ "question": "What is the path to the frontend-config.json file?", "follow_up": [{ "text": "./src/frontend-config.json", "mode": null }, { "text": "./config/frontend-config.json", "mode": null }, { "text": "./frontend-config.json", "mode": null }] }
@@ -14,7 +14,7 @@ Example: Asking with mode switch
1414

1515
const QUESTION_PARAMETER_DESCRIPTION = `Clear, specific question that captures the missing information you need`
1616

17-
const FOLLOW_UP_PARAMETER_DESCRIPTION = `Required list of 2-4 suggested responses; each suggestion must be a complete, actionable answer and may include a mode switch`
17+
const FOLLOW_UP_PARAMETER_DESCRIPTION = `Required array of 1-4 suggested responses, always an array even for a single suggestion; each suggestion must be a complete, actionable answer and may include a mode switch`
1818

1919
const FOLLOW_UP_TEXT_DESCRIPTION = `Suggested answer the user can pick`
2020

0 commit comments

Comments
 (0)