Skip to content

Commit b6e6122

Browse files
leon0399claude
andcommitted
fix(models): keep opencode-go off the streaming usage request
The OpenCode Go gateway answers stream_options.include_usage with a stream that has no finish reason and ends in a malformed cost chunk (anomalyco/opencode#42918). Replayed through llame's client, that stream ends with finishReason "error" and fails the Run despite complete text. Add an includeUsage switch to the shared Chat Completions client, defaulting to asking, and have the Go transport turn it off, with a test that fails when Go asks. Record the trap in apps/api/AGENTS.md and correct the changelog. Refs #1088 Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 446f379 commit b6e6122

5 files changed

Lines changed: 38 additions & 4 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,11 @@ _Reverse-chronological record of shipped work — features, fixes, and chores. N
66
Completions requests now ask the server for usage
77
(`stream_options.include_usage`), so a message's usage card shows input,
88
output, and reasoning tokens, and priced models record cost, instead of
9-
showing no counts. `opencode-go` requests share the client and now ask as
10-
well. The e2e model server now reports a streaming request's
9+
showing no counts. `opencode-go` requests do not ask: the OpenCode Go
10+
gateway answers the flag with a stream that has no finish reason
11+
([anomalyco/opencode#42918](https://github.com/anomalyco/opencode/issues/42918)),
12+
which would fail the Run. The e2e model server now reports a streaming
13+
request's
1114
usage only when the request asks, as real servers do. A new operator page
1215
covers vLLM-served models behind a LiteLLM gateway, including why its
1316
reasoning-effort levels can silently do nothing

‎apps/api/AGENTS.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,9 @@ the Anthropic API. `openai-codex` and `opencode-go` reject `baseUrl` because
100100
each endpoint is fixed in code: the Codex Responses endpoint, and the OpenCode
101101
Go gateway at `https://opencode.ai/zen/go/v1`, whose `type` executes the Chat
102102
Completions module with the Go transport (a fixed base URL, its own headers,
103-
redirect rejection, and a required credential). `accountId` is accepted by
103+
redirect rejection, a required credential, and no streaming usage request,
104+
because the gateway answers `stream_options.include_usage` with a stream that
105+
has no finish reason). `accountId` is accepted by
104106
`openai-codex` alone; every other variant rejects it. A `key` is optional for
105107
the two OpenAI wires and for Messages, where an absent or empty resolution is
106108
keyless; `openai-codex` requires nonblank `key` and `accountId`, and

‎apps/api/src/models/openai-completions-model-client.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,13 @@ export type OpenAICompletionsModelClientConfig = {
111111
providerOptions?: ProviderOptionRecord;
112112
/** Optional catalog output limit, forwarded as the request's `maxOutputTokens` setting. */
113113
maxOutputTokens?: number;
114+
/**
115+
* Whether streaming requests ask the server for usage
116+
* (`stream_options.include_usage`). Absent means they ask; a transport whose
117+
* server answers the flag with a stream the adapter cannot finish sets
118+
* `false` and records whatever usage the server sends unasked.
119+
*/
120+
includeUsage?: boolean;
114121
};
115122

116123
/**
@@ -386,7 +393,7 @@ export function createOpenAICompletionsModelClient(
386393
// Chat Completions servers report a streaming request's usage only when
387394
// the request asks (`stream_options.include_usage`). The adapter does not
388395
// ask by default, which leaves run usage accounting nothing to record.
389-
includeUsage: true,
396+
includeUsage: config.includeUsage ?? true,
390397
...(config.headers && { headers: config.headers }),
391398
...(config.fetch && { fetch: config.fetch }),
392399
});

‎apps/api/src/models/opencode-go-model-client.test.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -555,6 +555,23 @@ describe('createOpenCodeGoModelClient — the headers and body llame sends (desi
555555
}
556556
});
557557

558+
it('does not ask the gateway for streaming usage', async () => {
559+
const stub = serveFetch(streamResponse);
560+
try {
561+
const client = buildClient();
562+
563+
await expect(
564+
client.streamText({ chat: MAIN_CHAT, messages }).text,
565+
).resolves.toBe('done');
566+
567+
// The gateway answers `stream_options.include_usage` with a stream that
568+
// has no finish reason (anomalyco/opencode#42918), which fails the Run.
569+
expect(await requestBody(stub)).not.toHaveProperty('stream_options');
570+
} finally {
571+
stub.restore();
572+
}
573+
});
574+
558575
it('sends the same header set on the structured request', async () => {
559576
const stub = serveFetch(structuredResponse);
560577
try {

‎apps/api/src/models/opencode-go-model-client.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,11 @@ export function createOpenCodeGoModelClient(
104104
provider: 'opencode-go',
105105
headers: { [OPENCODE_GO_CLIENT_HEADER]: 'llame' },
106106
fetch: rejectRedirects(globalThis.fetch),
107+
// The gateway answers `stream_options.include_usage` with a stream that
108+
// carries no finish reason and ends in a malformed cost chunk, which
109+
// fails the Run (anomalyco/opencode#42918); without it the stream is
110+
// clean, so Go requests never ask.
111+
includeUsage: false,
107112
sessionHeader: (chat) => ({
108113
name: OPENCODE_GO_SESSION_HEADER,
109114
value: renderSessionValue(chat),

0 commit comments

Comments
 (0)