Skip to content

Commit 513c99a

Browse files
fix: treat non-empty finalText as success regardless of finish reason
Workaround for two open upstream bugs in OpenCode's `run --format json`: - anomalyco/opencode#26855 — race in cmd/run.ts: the run loop observes session.status=idle and exits before draining the final step_finish event to stdout, so lastReason stays undefined even though the model produced a complete response. Reproduced on prod with an explore workflow where a one-token "READY" reply from openai/gpt-5.5 came through as `text` but no step_finish ever followed. - anomalyco/opencode#27697 — JSON formatter drops post-tool-call assistant text, surfacing as step_finish.reason="tool-calls" alongside a terminal text completion that has no callable tool_use parts. Both manifest as "OpenCode exited cleanly with a final text response but the terminal accounting event never made it through." Until they land upstream, generalize OpencodeAccumulator.stopReason() to trust the text: any clean exit with non-empty finalText and no error events = success, regardless of lastReason. Mirror in chatResultFromTurn for the opencode-serve chat path, which has the same class of bug surfaced through `turn.finish` instead. Genuine truncation (model wanted more turns but got cut off) still leaves finalText empty and surfaces as the original error_<reason>. Cost is small: on affected runs we lose token/cost accounting from the missing step_finish, but we already have the answer. Revisit when anomalyco/opencode#26855 and anomalyco/opencode#27697 are resolved. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
1 parent b0947ed commit 513c99a

4 files changed

Lines changed: 58 additions & 27 deletions

File tree

‎src/engine/chat.test.ts‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -93,18 +93,26 @@ describe("handleChatMessage", () => {
9393
expect(r.stopReason).toBe("error_max_tokens");
9494
});
9595

96-
// gpt-5.5 quirk: OpenAI can return finish_reason="tool_calls" on a response
97-
// that ALSO carries a terminal text completion with no callable tool_use
98-
// parts. OpenCode passes that through as finish="tool-calls". Treat as
99-
// success when there's substantive text — otherwise good replies fail.
100-
it("treats finish=tool-calls + non-empty text as success (gpt-5.5 quirk)", async () => {
96+
// Workarounds for sst/opencode#26855 (final step_finish dropped) and
97+
// sst/opencode#27697 (post-tool-call text dropped). Both surface here as
98+
// turn.finish != "stop" even though turn.text holds a complete response.
99+
// Trust the text: any finish with substantive text is a successful turn.
100+
it("treats finish=tool-calls + non-empty text as success (gpt-5.5 quirk, #27697)", async () => {
101101
const { server } = stubServer(() => turn("here is your answer", { finish: "tool-calls" }));
102102
const r = await handleChatMessage("hi", "t", "alice", {} as any, { chatServer: server, ...baseDeps() }, baseConfig);
103103
expect(r.success).toBe(true);
104104
expect(r.stopReason).toBe("success");
105105
expect(r.text).toBe("here is your answer");
106106
});
107107

108+
it("treats finish=unknown + non-empty text as success (missing step_finish, #26855)", async () => {
109+
const { server } = stubServer(() => turn("READY", { finish: "unknown" }));
110+
const r = await handleChatMessage("hi", "t", "alice", {} as any, { chatServer: server, ...baseDeps() }, baseConfig);
111+
expect(r.success).toBe(true);
112+
expect(r.stopReason).toBe("success");
113+
expect(r.text).toBe("READY");
114+
});
115+
108116
it("still marks finish=tool-calls + empty text as error (genuine truncation)", async () => {
109117
const { server } = stubServer(() => turn("", { finish: "tool-calls", text: "" }));
110118
const r = await handleChatMessage("hi", "t", "alice", {} as any, { chatServer: server, ...baseDeps() }, baseConfig);

‎src/engine/chat.ts‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -277,15 +277,15 @@ function chatResultFromTurn(
277277
_userMessage: string,
278278
startTime: number,
279279
): ChatResultExt {
280-
// OpenAI gpt-5 family can return finish="tool-calls" on a response that
281-
// also contains a terminal text completion (no callable tool_use parts).
282-
// OpenCode passes that finish reason through verbatim. Treat it as success
283-
// when the model produced substantive text — otherwise we'd mark genuinely
284-
// good replies as failed. Mirrors the same heuristic in
285-
// OpencodeAccumulator.stopReason() (src/engine/opencode-executor.ts).
280+
// Mirror of OpencodeAccumulator.stopReason() in opencode-executor.ts —
281+
// see that comment for the full rationale, including upstream bugs
282+
// sst/opencode#26855 (final step_finish dropped before exit) and
283+
// sst/opencode#27697 (post-tool-call assistant text dropped). Both
284+
// manifest here as turn.finish being something other than "stop" even
285+
// though turn.text holds a complete response. Trust the text: any
286+
// non-error finish with substantive text is a successful turn.
286287
const success =
287-
turn.finish === "stop" ||
288-
(turn.finish === "tool-calls" && (turn.text?.length ?? 0) > 0);
288+
turn.finish === "stop" || (turn.text?.length ?? 0) > 0;
289289
const stopReason = success ? "success" : `error_${turn.finish.replace(/-/g, "_")}`;
290290
const turns = countSteps(turn.parts);
291291
return {

‎src/engine/opencode-executor.test.ts‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -71,20 +71,29 @@ describe("OpencodeAccumulator", () => {
7171
toolNoText.feed({ type: "step_finish", part: { reason: "tool-calls" } });
7272
expect(toolNoText.stopReason()).toBe("error_tool_calls");
7373

74-
// gpt-5.5 quirk: model emits final text AND finish_reason="tool_calls" in
75-
// the same response with no callable tool_use parts. OpenCode terminates
76-
// the loop with reason="tool-calls", but the run is actually complete —
77-
// treat as success when finalText is non-empty.
74+
// gpt-5.5 + sst/opencode#27697: model emits final text AND
75+
// finish_reason="tool_calls" in the same response with no callable
76+
// tool_use parts. OpenCode terminates the loop with reason="tool-calls",
77+
// but the run is actually complete — treat as success when finalText is
78+
// non-empty.
7879
const toolWithText = new Accumulator();
7980
toolWithText.feed({ type: "text", part: { text: "Here is the answer." } });
8081
toolWithText.feed({ type: "step_finish", part: { reason: "tool-calls" } });
8182
expect(toolWithText.stopReason()).toBe("success");
8283

84+
// sst/opencode#26855: OpenCode exits before draining the final
85+
// step_finish event, so lastReason is undefined even though a complete
86+
// text response was emitted. We have the answer — trust it.
87+
const textNoFinish = new Accumulator();
88+
textNoFinish.feed({ type: "text", part: { text: "READY" } });
89+
expect(textNoFinish.stopReason()).toBe("success");
90+
8391
const errored = new Accumulator();
8492
errored.feed({ type: "step_finish", part: { reason: "stop" } });
8593
errored.feed({ type: "error", error: { name: "x" } });
8694
expect(errored.stopReason()).toBe("error_api");
8795

96+
// Silent failure with no text and no events: still surfaces as unknown.
8897
const blank = new Accumulator();
8998
expect(blank.stopReason()).toBe("unknown");
9099
});

‎src/engine/opencode-executor.ts‎

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -211,23 +211,37 @@ class OpencodeAccumulator {
211211
/**
212212
* Map OpenCode's final reason → the legacy claude-style stopReason values
213213
* the dashboard and result-row writers already understand.
214+
*
215+
* Workarounds for two open upstream bugs in `opencode run --format json`:
216+
* - sst/opencode#26855 — the run loop sees session.status=idle and exits
217+
* before draining the final step_finish event to stdout, so lastReason
218+
* stays undefined even though the model produced a complete response.
219+
* - sst/opencode#27697 — the JSON formatter drops post-tool-call
220+
* assistant text on some model+stream combinations (we hit the gpt-5.5
221+
* variant where step_finish.reason="tool-calls" arrives alongside a
222+
* terminal text completion with no callable tool_use parts).
223+
*
224+
* Both manifest as "OpenCode exited cleanly with a final text response but
225+
* the terminal accounting event never made it through." Until those are
226+
* fixed upstream, treat any clean exit with non-empty finalText and no
227+
* error events as success — we already have the answer, the missing
228+
* step_finish just costs us token/cost accounting for that run.
229+
*
230+
* Revisit when sst/opencode#26855 and sst/opencode#27697 are resolved.
214231
*/
215232
stopReason(): string {
216233
if (this.errors.length > 0) return "error_api";
217-
if (!this.lastReason) return "unknown";
218234
if (this.lastReason === "stop") return "success";
219235
if (this.lastReason === "max_tokens" || this.lastReason === "length") {
220236
return "error_max_turns";
221237
}
222-
// OpenAI gpt-5 family can return finish_reason="tool_calls" on a response
223-
// that ALSO contains a terminal text completion (no actual tool_use parts
224-
// to execute). OpenCode then exits the loop with reason="tool-calls" even
225-
// though the model is done. If we have substantive final text and no API
226-
// errors, treat it as success. Genuine "tool-call truncation" cases still
227-
// surface as error_tool_calls because finalText will be empty.
228-
if (this.lastReason === "tool-calls" && this.finalText.length > 0) {
229-
return "success";
230-
}
238+
// Workaround for #26855 + #27697 (and the gpt-5.5 tool-calls quirk):
239+
// non-empty final text + no errors = the model finished, regardless of
240+
// whether a step_finish ever arrived. Genuine truncation (model wanted
241+
// more turns but got cut off) still leaves finalText empty and surfaces
242+
// as error_tool_calls / unknown.
243+
if (this.finalText.length > 0) return "success";
244+
if (!this.lastReason) return "unknown";
231245
return `error_${this.lastReason.replace(/-/g, "_")}`;
232246
}
233247

0 commit comments

Comments
 (0)