Skip to content

Commit ca0d7ff

Browse files
sunaba[bot]LLJY
authored andcommitted
fix(llm): prevent infinite loop when finish_reason missing in stream
In web mode some providers omit `finish_reason` on the final SSE chunk. `finishEvents()` left `reason` undefined in that case, the `step-finish` event was never emitted, and `processor.ts` returned `"continue"` indefinitely. When `finishReason` is undefined and no pending tools exist, default to `"stop"`. Pending (unfinished) tool calls still keep the error behavior to avoid emitting a false `"stop"` for interrupted tool calls. Fixes anomalyco#37855
1 parent 76361ca commit ca0d7ff

2 files changed

Lines changed: 19 additions & 1 deletion

File tree

‎packages/llm/src/protocols/openai-chat.ts‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -480,7 +480,10 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
480480
const finishEvents = (state: ParserState): ReadonlyArray<LLMEvent> => {
481481
const events: LLMEvent[] = []
482482
const hasToolCalls = state.toolCallEvents.length > 0
483-
const reason = state.finishReason === "stop" && hasToolCalls ? "tool-calls" : state.finishReason
483+
const hasPendingTools = Object.keys(state.tools).length > 0
484+
const reason = state.finishReason === "stop" && hasToolCalls
485+
? "tool-calls"
486+
: state.finishReason ?? (!hasPendingTools ? "stop" : undefined)
484487
const lifecycle = state.toolCallEvents.length ? Lifecycle.stepStart(state.lifecycle, events) : state.lifecycle
485488
events.push(...state.toolCallEvents)
486489
if (reason) Lifecycle.finish(lifecycle, events, { reason, usage: state.usage })

‎packages/llm/test/provider/openai-chat.test.ts‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -720,6 +720,21 @@ describe("OpenAI Chat route", () => {
720720
}),
721721
)
722722

723+
it.effect("emits step-finish with stop even without finish_reason in stream", () =>
724+
Effect.gen(function* () {
725+
const body = sseEvents(
726+
deltaChunk({ role: "assistant", content: "Hello" }),
727+
deltaChunk({ content: "!" }),
728+
)
729+
const events = Array.from(
730+
yield* LLMClient.stream(request).pipe(Stream.runCollect, Effect.provide(fixedResponse(body))),
731+
)
732+
expect(events.filter((e) => e.type === "step-finish")).toEqual([
733+
{ type: "step-finish", index: 0, reason: "stop", usage: undefined, providerMetadata: undefined },
734+
])
735+
}),
736+
)
737+
723738
it.effect("short-circuits the upstream stream when the consumer takes a prefix", () =>
724739
Effect.gen(function* () {
725740
// The body has more chunks than we'll consume. If `Stream.take(1)` did

0 commit comments

Comments
 (0)