Skip to content

fix(llminternal): guard nil first candidate in streaming aggregator - #1802

Merged
wojtas000 merged 2 commits into
google:mainfrom
PratikDhanaveFork:fix/stream-aggregator-nil-candidate
Oct 12, 2026
Merged

wojtas000 merged 2 commits into
google:mainfrom
PratikDhanaveFork:fix/stream-aggregator-nil-candidate

Conversation

@PratikDhanave

Copy link
Copy Markdown
Contributor

Summary

streamingResponseAggregator.ProcessResponse (internal/llminternal/stream_aggregator.go) checks len(genResp.Candidates) > 0 but then dereferences genResp.Candidates[0].FinishReason with no nil check. converters.Genai2LLMResponse — run on the same response one line above — already guards len(res.Candidates) > 0 && res.Candidates[0] != nil (with a comment that Vertex gemini-3* streams can send degenerate chunks). So a nil first candidate is reachable and panics the stream.

Change

Add && genResp.Candidates[0] != nil to the guard.

Verification

Regression test feeds a chunk with a nil first candidate — panics before the fix (nil pointer dereference), passes after. go test ./internal/llminternal/ green.

ProcessResponse checked len(genResp.Candidates) > 0 but then dereferenced
genResp.Candidates[0].FinishReason without a nil check. converters.Genai2LLMResponse,
run on the same response one line above, already guards
`len(res.Candidates) > 0 && res.Candidates[0] != nil` (a comment notes Vertex
gemini-3* streams can emit degenerate chunks), so a nil first candidate is
reachable and panics the stream. Add the same nil guard.

Add a regression test feeding a chunk with a nil first candidate.
@github-actions

Copy link
Copy Markdown

Thanks for the PR, @PratikDhanave.

One thing before review. Our contribution guide asks for an issue behind each change, and I could not find one linked here.

  • If an issue already exists, add Closes: #<number> or Related: #<number> to the description.
  • If not, open one — the link prefills the bug template with your PR title — then reference it the same way.

If this is a small doc or typo fix, ignore me. A maintainer can open the issue for you if you would rather.

@wojtas000 wojtas000 self-assigned this Oct 12, 2026

@wojtas000 wojtas000 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks. I reproduced the panic end to end: a Vertex-backend stream with a null candidate entry reaches ProcessResponse and panics at stream_aggregator.go:64 on main, and completes cleanly with this change. Small note for the record: the gemini-3* comment in converters.go is about chunks with no candidates, not nil ones, so I'll adjust the squash message.

@wojtas000
wojtas000 merged commit 1862aae into google:main Oct 12, 2026
14 checks passed
wolo-lab added a commit that referenced this pull request Oct 14, 2026
#785, #1719, #1804, #1802) (#1837)

* fix(utils): respect Nullable field on nil values during schema validation (#1566)

(cherry picked from commit 0e5797f)

* fix(llminternal): re-evaluate toolsets on every step in toolProcessor (#785)

(cherry picked from commit 707561d)

* fix(llminternal): omit foreign thoughts from agent context (#1719)

(cherry picked from commit a03e29e)

* fix(llminternal): correct misleading "global" in appendInstructions error (#1804)

(cherry picked from commit a3afd7f)

* fix(llminternal): guard nil first candidate in streaming aggregator (#1802)

(cherry picked from commit 1862aae)
wojtas000 added a commit to Sega757/adk-go that referenced this pull request Nov 4, 2026
The previous commit replaced the file with an older copy, which reverted
the JSON path parser (google#1630), the nil first-candidate guard (google#1802) and
usage metadata retention (google#1257), and did not compile.
wolo-lab added a commit that referenced this pull request Nov 6, 2026
…or (#2051)

* perf(llminternal): use strings.Builder in streaming response aggregator

* perf(llminternal): use strings.Builder for the streaming text buffer

* revert(llminternal): restore stream_aggregator.go from main

The previous commit replaced the file with an older copy, which reverted
the JSON path parser (#1630), the nil first-candidate guard (#1802) and
usage metadata retention (#1257), and did not compile.

* perf(llminternal): use strings.Builder for the streaming text buffer

Appending each streamed chunk with += recopied the whole text received so
far, so aggregating a long reply was quadratic in its length. The test
checks that doubling the chunk count about doubles the bytes allocated.

---------

Co-authored-by: wojtasjan <[email protected]>
Co-authored-by: wolo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants