Skip to content

Commit b7cee94

Browse files
author
tony
committed
fix(core): recover sessions after payload size rejection
1 parent f72a103 commit b7cee94

9 files changed

Lines changed: 157 additions & 20 deletions

File tree

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,7 +182,7 @@ const table = sqliteTable("session", {
182182
- Reusing a Session ID adopts the existing Session. Reusing a user or synthetic inbox item ID is idempotent when Session and type match: the first admission wins and the retried payload, metadata, and delivery mode are ignored, whether the item is still pending or already delivered (reconciled from the projected message without retained enqueue history). Cross-Session or cross-type reuse fails. Control items keep their operation-specific conflict behavior.
183183
- Keep `SessionExecution` process-global and Session-ID based. Its local implementation owns the process-local Session coordinator and discovers placement through `SessionStore` plus `LocationServiceMap.get(session.location)` only when a drain starts; no layer should take a Session ID. V2 interruption targets the active process-local ownership chain for that Session; interruption of a known but idle or locally unowned Session is a no-op, while the public API rejects an unknown Session.
184184
- Keep `SessionRunner`, model resolution, tool registry, permissions, and filesystem Location-scoped. Omitted `Location.workspaceID` means implicit-local placement; explicit workspace identity remains reserved for future placement semantics.
185-
- Preserve one explicit `llm.stream(request)` call per Physical Attempt and reload projected history before durable continuation. A logical Step may use generic pre-output retries, one full-context retry after continuation rejection, incomplete-stream continuation, or one overflow-compaction rebuild. Generic retries retain the logical step number and do not consume another agent-step allowance. Do not delegate orchestration to an in-memory tool loop.
185+
- Preserve one explicit `llm.stream(request)` call per Physical Attempt and reload projected history before durable continuation. A logical Step may use generic pre-output retries, one full-context retry after continuation rejection, incomplete-stream continuation, or one overflow-compaction rebuild. Both token-window and request-payload rejections may trigger that rebuild before durable output; payload recovery starts compaction without inline media. Generic retries retain the logical step number and do not consume another agent-step allowance. Do not delegate orchestration to an in-memory tool loop.
186186
- Keep local Session drains process-local until clustering is implemented. `SessionRunCoordinator` joins explicit same-Session resumes, coalesces prompt wakeups, and allows different Sessions to run concurrently. A write-ahead execution claim marks a process-local busy period for restart recovery: terminal completion, failure, or user interruption releases it, while shutdown interruption and process death preserve it. Startup recovery resumes claimed top-level Sessions with durable per-execution attempt accounting. The claim is a recovery marker, not clustered ownership, fencing, or an exactly-once guarantee.
187187
- Keep provider-specific native compaction mechanisms in `@opencode/ai` behind `LLMClient.compact`. `SessionCompaction` chooses a summary or native compaction from the model's `compaction` setting and owns route provenance, request shrinking, the retry policy, interruption, usage accounting, and checkpoint persistence.
188188
- Keep delivery vocabulary explicit. Prompts steer by default. At safe step boundaries, steered compaction takes priority up to the first steered move control; other steers retain enqueue order. At an idle boundary, steers take priority; otherwise exactly one queued item delivers before the runner reevaluates continuation. Inbox items may be cancelled or changed between queue and steer before delivery. Promoting new user input resets the selected agent's step allowance; a batch of steers resets it once.

‎packages/ai/src/index.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ export { ImageClient } from "./image-client.js"
44
export { Auth } from "./route/auth.js"
55
export { Provider } from "./provider.js"
66
export { ProviderPackage } from "./provider-package.js"
7-
export { isContextOverflow, isContextOverflowFailure, isRetryable } from "./provider-error.js"
7+
export { isContextOverflow, isContextOverflowFailure, isPayloadTooLargeFailure, isRetryable } from "./provider-error.js"
88
export type {
99
RouteLanguageModelInput,
1010
RouteRoutedLanguageModelInput,

‎packages/ai/src/provider-error.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,11 @@ export const isContextOverflowFailure = (failure: unknown) =>
7474
? failure.reason._tag === "InvalidRequest" && failure.reason.classification === "context-overflow"
7575
: Schema.is(ProviderErrorEvent)(failure) && failure.classification === "context-overflow"
7676

77+
export const isPayloadTooLargeFailure = (failure: unknown) =>
78+
failure instanceof AIError
79+
? failure.reason._tag === "InvalidRequest" && failure.reason.classification === "payload-too-large"
80+
: Schema.is(ProviderErrorEvent)(failure) && failure.classification === "payload-too-large"
81+
7782
/**
7883
* Whether a failed call may succeed when sent again: rate limits, provider-side failures, transport failures that did
7984
* not deliver an accepted write, and unrecognized failures. Callers decide which calls are safe to repeat.

‎packages/ai/test/provider-error.test.ts‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { describe, expect, test } from "bun:test"
2-
import { isContextOverflow } from "../src/index.js"
2+
import { AIError, InvalidRequestError, LLMEvent, isContextOverflow, isPayloadTooLargeFailure } from "../src/index.js"
33
import { classifyProviderFailure, providerErrorMessage } from "../src/provider-error.js"
44

55
describe("provider error classification", () => {
@@ -45,6 +45,30 @@ describe("provider error classification", () => {
4545
)
4646
})
4747

48+
test("recognizes payload rejection errors and stream events without treating other failures as size errors", () => {
49+
const reason = classifyProviderFailure({
50+
message: "Gateway request body exceeds the configured limit.",
51+
status: 413,
52+
rawBody: '{"error":{"code":"proxy_error","status":413}}',
53+
})
54+
expect(isPayloadTooLargeFailure(new AIError({ reason }))).toBe(true)
55+
expect(
56+
isPayloadTooLargeFailure(
57+
LLMEvent.providerError({
58+
message: "Too large",
59+
classification: "payload-too-large",
60+
}),
61+
),
62+
).toBe(true)
63+
expect(isPayloadTooLargeFailure(new AIError({ reason: new InvalidRequestError({ message: "Invalid" }) }))).toBe(
64+
false,
65+
)
66+
expect(
67+
isPayloadTooLargeFailure(LLMEvent.providerError({ message: "Too long", classification: "context-overflow" })),
68+
).toBe(false)
69+
expect(isPayloadTooLargeFailure(undefined)).toBe(false)
70+
})
71+
4872
test("does not classify rate limits as context overflow", () => {
4973
const messages = [
5074
"Throttling error: Too many tokens, please wait before trying again.",

‎packages/core/src/session/compaction.ts‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,8 @@ export type Editor = {
4848
}
4949

5050
export type Trigger =
51-
/** `overflow`: the provider just rejected this context as too long. */
52-
| { readonly reason: "auto" | "overflow"; readonly context: SessionContext.Loaded }
51+
/** `overflow`: token window rejection; `payload`: request body size rejection. */
52+
| { readonly reason: "auto" | "overflow" | "payload"; readonly context: SessionContext.Loaded }
5353
/** `inputID` is the `/compact` inbox item, whose message shows the outcome. */
5454
| { readonly reason: "manual"; readonly context: SessionContext.Loaded; readonly inputID: SessionMessage.ID }
5555

@@ -383,7 +383,8 @@ export const layer = Layer.effect(
383383
const policy = yield* SessionRunnerRetry.policy(context.session.id)
384384
let rejections = 0
385385
let rejected: number | undefined
386-
let asText = false
386+
// A primary request already rejected for its bytes must not resend the same inline media to summarize it.
387+
let asText = trigger.reason === "payload"
387388

388389
while (true) {
389390
const fits = !asText && estimateRequest(prepared.request) <= target

‎packages/core/src/session/runner/llm.ts‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -259,13 +259,14 @@ const layer = Layer.effect(
259259
retry: proposed,
260260
}),
261261
recoverContinuation,
262-
recoverOverflow: Effect.suspend(() =>
263-
recoverOverflow
264-
? compaction
265-
.compact({ reason: "overflow", context: loaded })
266-
.pipe(Effect.map((result) => result.status === "completed"))
267-
: Effect.succeed(false),
268-
),
262+
recoverOverflow: (reason) =>
263+
Effect.suspend(() =>
264+
recoverOverflow
265+
? compaction
266+
.compact({ reason, context: loaded })
267+
.pipe(Effect.map((result) => result.status === "completed"))
268+
: Effect.succeed(false),
269+
),
269270
})
270271
const completed = yield* SessionStep.Outcome.$match(outcome, {
271272
Completed: (outcome) => Effect.succeed(outcome.needsContinuation),

‎packages/core/src/session/runner/step.ts‎

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import {
66
LLMClient,
77
LLMEvent,
88
isContextOverflowFailure,
9+
isPayloadTooLargeFailure,
910
type ProviderErrorEvent,
1011
type ToolCall,
1112
} from "@opencode/ai"
@@ -55,13 +56,16 @@ interface Input {
5556
) => Effect.Effect<{ readonly retry: false } | SessionRunnerRetry.Decision>
5657
readonly recoverContinuation: boolean
5758
/** The runner owns compaction policy; the attempt invokes it only before durable output. */
58-
readonly recoverOverflow: Effect.Effect<boolean>
59+
readonly recoverOverflow: (reason: "overflow" | "payload") => Effect.Effect<boolean>
5960
}
6061

6162
const TOOLS_INTERRUPTED = { type: "aborted", message: "Tool execution interrupted" } as const
6263
const STEP_INTERRUPTED = { type: "aborted", message: "Step interrupted" } as const
6364
const RESULT_MISSING = { type: "tool.result-missing", message: "Provider did not return a tool result" } as const
6465

66+
const isRequestOverflowFailure = (failure: unknown) =>
67+
isContextOverflowFailure(failure) || isPayloadTooLargeFailure(failure)
68+
6569
/** Captures Location-scoped dependencies without introducing another service or execution loop. */
6670
export const make = Effect.gen(function* () {
6771
const bus = yield* Bus.Service
@@ -107,7 +111,7 @@ export const make = Effect.gen(function* () {
107111
if (overflowFailure || publisher.hasProviderError()) return
108112
if (
109113
LLMEvent.is.providerError(event) &&
110-
isContextOverflowFailure(event) &&
114+
isRequestOverflowFailure(event) &&
111115
!publisher.record().outputStarted
112116
) {
113117
overflowFailure = event
@@ -147,12 +151,13 @@ export const make = Effect.gen(function* () {
147151
const overflow = overflowFailure ?? streamFailure
148152
if (
149153
!publisher.record().outputStarted &&
150-
isContextOverflowFailure(overflow) &&
151-
(yield* restore(input.recoverOverflow))
154+
isRequestOverflowFailure(overflow) &&
155+
(yield* restore(input.recoverOverflow(isPayloadTooLargeFailure(overflow) ? "payload" : "overflow")))
152156
) {
153-
yield* Effect.logWarning("provider rejected the request as too long; compacting", {
157+
yield* Effect.logWarning("provider rejected the request size; compacted before retrying", {
154158
sessionID: input.sessionID,
155159
model: input.model.ref,
160+
reason: isPayloadTooLargeFailure(overflow) ? "payload" : "overflow",
156161
message: overflow?.message,
157162
})
158163
return Outcome.Compacted()
@@ -179,7 +184,7 @@ export const make = Effect.gen(function* () {
179184
)
180185
return Outcome.RecoverFull()
181186
const retry =
182-
llmFailure && llmError && !isContextOverflowFailure(llmFailure)
187+
llmFailure && llmError && !isRequestOverflowFailure(llmFailure)
183188
? yield* restore(
184189
input.retry(
185190
llmFailure,

‎packages/core/test/session-runner.test.ts‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3268,6 +3268,107 @@ describe("SessionRunnerLLM", () => {
32683268
expect(yield* s.context).toMatchObject([{ type: "compaction" }, { type: "assistant", finish: "stop" }])
32693269
})
32703270

3271+
for (const raw of [false, true]) {
3272+
scenario(`recovers once after a ${raw ? "raw" : "streamed"} payload-too-large rejection`, function* (s) {
3273+
yield* setupOverflowRecovery(s)
3274+
yield* s.llm.push(
3275+
raw
3276+
? Stream.fail(payloadTooLarge())
3277+
: [LLMEvent.providerError({ message: "Too large", classification: "payload-too-large" })],
3278+
TestLLM.text("## Objective\n- Recover payload", "text-summary"),
3279+
TestLLM.text("Recovered", "text-final"),
3280+
)
3281+
yield* s.runPrompt("Continue")
3282+
3283+
expect(s.requests).toHaveLength(3)
3284+
expect(yield* s.context).toMatchObject([
3285+
{ type: "compaction", summary: "## Objective\n- Recover payload" },
3286+
{ type: "assistant", finish: "stop" },
3287+
])
3288+
expect(yield* recordedEventTypes(sessionID)).not.toContain("session.step.failed.1")
3289+
yield* replaySessionProjection(sessionID)
3290+
expect(yield* s.context).toMatchObject([{ type: "compaction" }, { type: "assistant", finish: "stop" }])
3291+
})
3292+
}
3293+
3294+
scenario("omits inline media from the first payload recovery compaction and preserves stored history", function* (s) {
3295+
const image = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII="
3296+
yield* s.session.prompt({
3297+
sessionID,
3298+
text: "Inspect the screenshot",
3299+
files: [{ uri: `data:image/png;base64,${image}` }],
3300+
resume: false,
3301+
})
3302+
yield* s.llm.push(TestLLM.text("The screenshot shows a button", "inspection"))
3303+
yield* s.resume
3304+
s.requests.length = 0
3305+
yield* s.llm.push(
3306+
Stream.fail(payloadTooLarge()),
3307+
TestLLM.text("## Objective\n- Inspect the button", "summary"),
3308+
TestLLM.text("Recovered", "recovered"),
3309+
)
3310+
yield* s.runPrompt("Continue")
3311+
3312+
expect(s.requests).toHaveLength(3)
3313+
expect(s.requests[0]?.messages.some((m) => m.content.some((p) => p.type === "media"))).toBeTrue()
3314+
expect(s.requests[1]?.messages.every((m) => m.content.every((p) => p.type === "text"))).toBeTrue()
3315+
expect(JSON.stringify(s.requests[1])).not.toContain(image)
3316+
expect(JSON.stringify(s.requests[1])).toContain("The screenshot shows a button")
3317+
expect(JSON.stringify(yield* s.messages)).toContain(image)
3318+
})
3319+
3320+
scenario("does not recover payload rejection with automatic compaction disabled", function* (s) {
3321+
yield* setupOverflowRecovery(s)
3322+
const compaction = yield* SessionCompaction.Service
3323+
yield* compaction.transform((editor) => editor.configure({ auto: false }))
3324+
yield* s.llm.push(Stream.fail(payloadTooLarge()))
3325+
expect((yield* s.runPrompt("Continue").pipe(Effect.flip)).message).toBe("Too large")
3326+
expect(s.requests).toHaveLength(1)
3327+
expect(yield* s.context).not.toContainEqual(expect.objectContaining({ type: "compaction" }))
3328+
})
3329+
3330+
scenario("stops after a second payload rejection instead of compacting forever", function* (s) {
3331+
yield* setupOverflowRecovery(s)
3332+
yield* s.llm.push(
3333+
Stream.fail(payloadTooLarge()),
3334+
TestLLM.text("## Objective\n- Recover once", "summary"),
3335+
Stream.fail(payloadTooLarge()),
3336+
)
3337+
expect((yield* s.runPrompt("Continue").pipe(Effect.flip)).message).toBe("Too large")
3338+
expect(s.requests).toHaveLength(3)
3339+
expect(yield* s.context).toMatchObject([
3340+
{ type: "compaction", status: "completed" },
3341+
{ type: "assistant", finish: "error", error: { message: "Too large" } },
3342+
])
3343+
})
3344+
3345+
scenario("does not recover payload rejection after durable output", function* (s) {
3346+
yield* setupOverflowRecovery(s)
3347+
yield* s.llm.push([
3348+
LLMEvent.textStart({ id: "partial" }),
3349+
LLMEvent.textDelta({ id: "partial", text: "Partial" }),
3350+
LLMEvent.textEnd({ id: "partial" }),
3351+
LLMEvent.providerError({ message: "Too large", classification: "payload-too-large" }),
3352+
])
3353+
expect((yield* s.runPrompt("Continue").pipe(Effect.flip)).message).toBe("Too large")
3354+
expect(s.requests).toHaveLength(1)
3355+
expect(yield* s.context).not.toContainEqual(expect.objectContaining({ type: "compaction" }))
3356+
})
3357+
3358+
scenario("preserves the payload error when recovery compaction fails", function* (s) {
3359+
yield* setupOverflowRecovery(s)
3360+
yield* s.llm.push(Stream.fail(payloadTooLarge()), [LLMEvent.providerError({ message: "Summary unavailable" })])
3361+
expect((yield* s.runPrompt("Continue").pipe(Effect.flip)).message).toBe("Too large")
3362+
expect(s.requests).toHaveLength(2)
3363+
expect(yield* s.context).toContainEqual(
3364+
expect.objectContaining({
3365+
type: "compaction",
3366+
status: "failed",
3367+
error: { type: "provider.error", message: "Summary unavailable" },
3368+
}),
3369+
)
3370+
})
3371+
32713372
scenario("refreshes preparation after overflow compaction without promoting new input", function* (s) {
32723373
yield* setupOverflowRecovery(s)
32733374

‎packages/core/test/session-step.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ for (const fixture of [
118118
retry: (_cause, _error, retry) =>
119119
Effect.succeed(retry ? { retry: true, attempt: 2, delay: 0 } : { retry: false }),
120120
recoverContinuation: true,
121-
recoverOverflow: Effect.succeed(false),
121+
recoverOverflow: () => Effect.succeed(false),
122122
})
123123
.pipe(Effect.exit)
124124
expect(Exit.isSuccess(result)).toBe(fixture.finish === "stop")

0 commit comments

Comments
 (0)