Skip to content

Commit 9d4d6f3

Browse files
byapparovclaude
andcommitted
fix(run): give schema runs a corrective turn after a permission rejection (#135)
In --output-schema mode a step that stopped on a headless permission auto-reject ended the run as structured_output failed/missing with the repair budget unspent. Treat it like a prose-only finish: it counts as an attempt, emits structured_output_rejected with the fixed `denied` diagnostic, and the next turn forces StructuredOutput with a reminder that names the denied tools (never their arguments). The existing budget still bounds it; exhaustion keeps reason `missing` and exit 3. Runs without a schema still stop on a rejection. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent cd736e2 commit 9d4d6f3

5 files changed

Lines changed: 154 additions & 36 deletions

File tree

‎EVENTS.md‎

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -580,7 +580,8 @@ value in the terminal event below.
580580
### `structured_output_rejected`
581581

582582
Emitted once per rejected StructuredOutput attempt, including an unusable result
583-
file and a model turn that finishes in prose without a final tool call:
583+
file, a model turn that finishes in prose without a final tool call, and a step
584+
stopped by a headless permission rejection:
584585

585586
```json
586587
{
@@ -618,6 +619,14 @@ asks it to fix the file at the JSON-encoded path. Malformed tool arguments (not
618619
A prose-only finish emits the fixed diagnostic
619620
`{ "path": "", "keyword": "missing", "message": "write the final result as JSON to a file in the working directory, then call StructuredOutput with its path" }`
620621
and receives an ephemeral reminder before the next turn while budget remains.
622+
A step that stops because a tool call's permission was auto-rejected (headless
623+
mode has no user to approve; see `permission_rejected`) is handled the same way.
624+
It emits the fixed diagnostic
625+
`{ "path": "", "keyword": "denied", "message": "a tool call was denied; write the final result as JSON to a file in the working directory, then call StructuredOutput with its path" }`.
626+
The reminder names the denied tools, not their arguments, and asks the model to
627+
finish with what it already has. Both corrective turns force the StructuredOutput
628+
tool choice. Without `--output-schema`, a permission rejection
629+
still ends the run.
621630
Other tools remain usable until a valid final result; their outputs are not final
622631
results. Only the `{ path }` tool schema is sent to the provider; local validation
623632
of the file always uses the canonical schema.
@@ -638,19 +647,21 @@ or:
638647
```
639648

640649
`attempts` counts StructuredOutput calls, including rejected/unparseable ones,
641-
and prose-only finishes without a final call. Other tool calls do not count.
650+
prose-only finishes without a final call, and steps stopped by a permission
651+
rejection. Other tool calls do not count.
642652
Compaction turns do not consume a missing-result attempt. Discarded provider
643653
streams restore the budget and captured result; their rejection events remain
644654
as telemetry. A valid call in a parallel step wins regardless of call order if
645655
that step started with budget remaining. The attempt count includes every call
646656
in that step and can exceed `maxAttempts`; the N+1 bound applies to corrective
647657
turns.
648658
When the budget is spent, the last rejection determines the reason: `missing`
649-
for a prose-only finish, otherwise `exhausted`. Failure reasons are:
659+
for a prose-only finish or a permission rejection, otherwise `exhausted`.
660+
Failure reasons are:
650661

651662
- `exhausted`: invalid or unparseable calls spent the corrective budget.
652-
- `missing`: prose finishes spent the budget, or the session ended normally
653-
without an accepted result (including headless permission rejection).
663+
- `missing`: prose finishes or permission rejections spent the budget, or the
664+
session ended normally without an accepted result.
654665
- `step_limit`: the agent reached its configured step cap without a result.
655666
- `aborted`: the invocation was cancelled, including SIGINT/SIGTERM.
656667
- `error`: a provider or session error, including stream idle timeout or an
@@ -664,4 +675,5 @@ Exit codes are **0** on accepted results, **2** for schema configuration errors,
664675
**3** for exhausted attempts, missing output and step limits. Provider errors and
665676
timeouts keep **1**; SIGINT and SIGTERM keep **130** and **143**.
666677
`StructuredOutputError.data.retries` records the number of corrective
667-
attempts used (initial attempt excluded), including prose-only corrective turns.
678+
attempts used (initial attempt excluded), including corrective turns after a
679+
prose-only finish or a permission rejection.

‎packages/cli/src/session/processor.ts‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,8 @@ export namespace SessionProcessor {
3434
abort: AbortSignal
3535
}) {
3636
const toolcalls: Record<string, MessageV2.ToolPart> = {}
37-
let blocked = false
37+
// Tools whose rejected permission stopped the loop.
38+
const blocked = new Set<string>()
3839
let attempt = 0
3940
let needsCompaction = false
4041
const retried: MessageV2.ToolPart[] = []
@@ -56,6 +57,9 @@ export namespace SessionProcessor {
5657
get message() {
5758
return input.assistantMessage
5859
},
60+
get blocked() {
61+
return [...blocked]
62+
},
5963
partFromToolCall(toolCallID: string) {
6064
return toolcalls[toolCallID]
6165
},
@@ -285,10 +289,11 @@ export namespace SessionProcessor {
285289
})
286290

287291
if (
288-
value.error instanceof PermissionNext.RejectedError ||
289-
value.error instanceof Question.RejectedError
292+
shouldBreak &&
293+
(value.error instanceof PermissionNext.RejectedError ||
294+
value.error instanceof Question.RejectedError)
290295
) {
291-
blocked = shouldBreak
296+
blocked.add(match.tool)
292297
}
293298
delete toolcalls[value.toolCallId]
294299
}
@@ -528,7 +533,7 @@ export namespace SessionProcessor {
528533
// status; headless consumers stop reading when they observe idle.
529534
await Session.updateMessage(input.assistantMessage)
530535
if (needsCompaction) return "compact"
531-
if (blocked) return "stop"
536+
if (blocked.size) return "stop"
532537
if (input.assistantMessage.error) return "stop"
533538
return "continue"
534539
}

‎packages/cli/src/session/prompt.ts‎

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,8 @@ IMPORTANT:
5858
const STRUCTURED_OUTPUT_MISSING =
5959
"write the final result as JSON to a file in the working directory, then call StructuredOutput with its path"
6060

61+
const STRUCTURED_OUTPUT_DENIED = `a tool call was denied; ${STRUCTURED_OUTPUT_MISSING}`
62+
6163
const STRUCTURED_OUTPUT_SYSTEM_PROMPT = `IMPORTANT: The user has requested structured output. Write your final result as JSON to a file in the working directory, then call the StructuredOutput tool with that file's path. Do NOT respond with plain text.`
6264

6365
export namespace SessionPrompt {
@@ -84,6 +86,22 @@ export namespace SessionPrompt {
8486
return !hasToolCalls(await load())
8587
}
8688

89+
// A schema turn that stops without a result costs one attempt and earns a
90+
// corrective reminder: a prose-only finish, or a headless permission
91+
// rejection (#135). The reminder names the denied tools, never their input.
92+
async function missingResult(processor: SessionProcessor.Info) {
93+
if (processor.blocked.length && !processor.message.error)
94+
return {
95+
diagnostic: { path: "", keyword: "denied", message: STRUCTURED_OUTPUT_DENIED },
96+
reminder: `Permission denied for ${processor.blocked.join(", ")}. Using what you already have, ${STRUCTURED_OUTPUT_MISSING}`,
97+
}
98+
if (await missingStructuredOutput(processor.message, () => MessageV2.parts(processor.message.id)))
99+
return {
100+
diagnostic: { path: "", keyword: "missing", message: STRUCTURED_OUTPUT_MISSING },
101+
reminder: STRUCTURED_OUTPUT_MISSING,
102+
}
103+
}
104+
87105
const state = Instance.state(
88106
() => {
89107
const data: Record<
@@ -329,7 +347,7 @@ export namespace SessionPrompt {
329347
let structuredOutput: unknown | undefined
330348
let attempts = 0
331349
let rejected = 0
332-
let reminder = false
350+
let reminder: string | undefined
333351
let user: string | undefined
334352
let contract:
335353
| {
@@ -383,7 +401,7 @@ export namespace SessionPrompt {
383401
: undefined
384402
attempts = 0
385403
rejected = 0
386-
reminder = false
404+
reminder = undefined
387405
structuredOutput = undefined
388406
}
389407
const lastAssistantMsg = msgs.findLast((msg) => msg.info.id === lastAssistant?.id)
@@ -799,7 +817,7 @@ export namespace SessionPrompt {
799817
? [
800818
{
801819
role: "user" as const,
802-
content: `<system-reminder>${STRUCTURED_OUTPUT_MISSING}</system-reminder>`,
820+
content: `<system-reminder>${reminder}</system-reminder>`,
803821
},
804822
]
805823
: []),
@@ -859,11 +877,9 @@ export namespace SessionPrompt {
859877
continue
860878
}
861879

862-
reminder =
863-
!!contract && (await missingStructuredOutput(processor.message, () => MessageV2.parts(processor.message.id)))
864-
if (reminder) {
865-
await rejection([{ path: "", keyword: "missing", message: STRUCTURED_OUTPUT_MISSING }])
866-
}
880+
const missing = contract ? await missingResult(processor) : undefined
881+
reminder = missing?.reminder
882+
if (missing) await rejection([missing.diagnostic])
867883

868884
if (contract && !processor.message.error && rejected > contract.format.retryCount) {
869885
processor.message.error = new MessageV2.StructuredOutputError({

‎packages/cli/test/cli/run-output-schema.test.ts‎

Lines changed: 71 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,6 @@ async function run(
2121
queued?: boolean
2222
attach?: boolean
2323
locked?: boolean
24-
denied?: boolean
2524
} = {},
2625
) {
2726
await using tmp = await tmpdir()
@@ -71,12 +70,14 @@ async function run(
7170
index: 0,
7271
id: `call_${requests.length}`,
7372
type: "function",
74-
function: options.denied
75-
? { name: "read", arguments: JSON.stringify({ filePath: "/outside-fixture.txt" }) }
76-
: {
77-
name: "StructuredOutput",
78-
arguments: pathArgs(tmp.path, args, `so_${requests.length}.json`),
79-
},
73+
// "denied" reads outside the run directory, which headless mode auto-rejects.
74+
function:
75+
args === "denied"
76+
? { name: "read", arguments: JSON.stringify({ filePath: "/outside-fixture.txt" }) }
77+
: {
78+
name: "StructuredOutput",
79+
arguments: pathArgs(tmp.path, args, `so_${requests.length}.json`),
80+
},
8081
},
8182
],
8283
},
@@ -85,7 +86,7 @@ async function run(
8586
],
8687
},
8788
{
88-
choices: [{ index: 0, delta: {}, finish_reason: options.denied ? "tool-calls" : "stop" }],
89+
choices: [{ index: 0, delta: {}, finish_reason: args === "denied" ? "tool-calls" : "stop" }],
8990
usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },
9091
},
9192
]
@@ -762,22 +763,78 @@ test("JSON mode without output-result carries the accepted value only in termina
762763
])
763764
}, 25000)
764765

765-
test("headless permission rejection reports missing output with no session error", async () => {
766+
const DENIED_REMINDER =
767+
"<system-reminder>Permission denied for read. Using what you already have, write the final result as JSON to a file in the working directory, then call StructuredOutput with its path</system-reminder>"
768+
769+
test("headless permission rejection gets a corrective turn that can still succeed", async () => {
766770
const result = await run({
767-
denied: true,
771+
inputs: ["denied", '{"result":"accepted"}'],
772+
flags: ["--output-schema", "schema.json", "--output-result", "result.json"],
773+
})
774+
expect(result.exit, result.stderr + result.stdout).toBe(0)
775+
expect(result.requests).toHaveLength(2)
776+
expect(result.requests.map((request) => request.tool_choice)).toEqual([
777+
"required",
778+
{ type: "function", function: { name: "StructuredOutput" } },
779+
])
780+
const messages = result.requests[1].messages as { role: string; content: unknown }[]
781+
expect(messages.filter((message) => JSON.stringify(message.content).includes("Permission denied"))).toEqual([
782+
{ role: "user", content: DENIED_REMINDER },
783+
])
784+
expect(result.result).toBe(JSON.stringify({ result: "accepted" }, null, 2) + "\n")
785+
expect(result.events.filter((event) => event.type === "permission_rejected")).toHaveLength(1)
786+
expect(result.events.filter((event) => event.type === "session_error")).toEqual([])
787+
expect(result.events.filter((event) => event.type === "structured_output_rejected")).toMatchObject([
788+
{
789+
attempt: 1,
790+
maxAttempts: 3,
791+
errors: [
792+
{
793+
path: "",
794+
keyword: "denied",
795+
message:
796+
"a tool call was denied; write the final result as JSON to a file in the working directory, then call StructuredOutput with its path",
797+
},
798+
],
799+
},
800+
])
801+
expect(result.events.filter((event) => event.type === "structured_output")).toMatchObject([
802+
{ status: "accepted", attempts: 2, value: { result: "accepted" } },
803+
])
804+
}, 25000)
805+
806+
test("repeated headless permission rejections spend the budget as missing output", async () => {
807+
const result = await run({
808+
inputs: ["denied"],
768809
existing: "untouched",
769810
flags: ["--output-schema", "schema.json", "--output-result", "result.json"],
770811
})
771812
expect(result.exit, result.stderr + result.stdout).toBe(3)
772-
expect(result.requests).toHaveLength(1)
813+
expect(result.requests).toHaveLength(3)
773814
expect(result.result).toBe("untouched")
774-
expect(result.events.filter((event) => event.type === "permission_rejected")).toHaveLength(1)
775-
expect(result.events.filter((event) => event.type === "session_error")).toEqual([])
815+
expect(result.events.filter((event) => event.type === "permission_rejected")).toHaveLength(3)
816+
expect(
817+
result.events
818+
.filter((event) => event.type === "structured_output_rejected")
819+
.map((event) => [event.attempt, (event.errors as { keyword: string }[])[0].keyword]),
820+
).toEqual([
821+
[1, "denied"],
822+
[2, "denied"],
823+
[3, "denied"],
824+
])
776825
expect(result.events.filter((event) => event.type === "structured_output")).toMatchObject([
777-
{ status: "failed", reason: "missing", attempts: 0 },
826+
{ status: "failed", reason: "missing", attempts: 3 },
778827
])
779828
}, 25000)
780829

830+
test("plain run still stops after a headless permission rejection", async () => {
831+
const result = await run({ inputs: ["denied"], flags: [] })
832+
expect(result.exit, result.stderr + result.stdout).toBe(0)
833+
expect(result.requests).toHaveLength(1)
834+
expect(result.events.filter((event) => event.type === "permission_rejected")).toHaveLength(1)
835+
expect(result.events.filter((event) => String(event.type).startsWith("structured_output"))).toEqual([])
836+
}, 25000)
837+
781838
test("schema config NDJSON clips the specific compilation cause", async () => {
782839
const result = await run({ content: JSON.stringify({ type: "object", ["unknown".repeat(100)]: true }) })
783840
expect(result.exit).toBe(2)

‎packages/cli/test/session/output-schema.test.ts‎

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -612,12 +612,40 @@ for (const parallel of [
612612
}, 15000)
613613
}
614614

615-
test("permission denial without a provider error reports missing output", async () => {
615+
test("permission rejection with zero retries spends the only attempt as missing output", async () => {
616616
const result = await run(["unused"], 0, { denied: true })
617617
expect(result.requests, JSON.stringify(result.info)).toHaveLength(1)
618-
expect(result.info.error).toBeUndefined()
618+
expect(result.info.error).toMatchObject({
619+
name: "StructuredOutputError",
620+
data: { message: "Model did not produce structured output", retries: 0 },
621+
})
619622
expect(result.info.structured).toBeUndefined()
620-
expect(result.outcomes).toEqual([{ status: "failed", reason: "missing", attempts: 0 }])
623+
expect(result.rejected).toMatchObject([{ attempt: 1, maxAttempts: 1, errors: [{ path: "", keyword: "denied" }] }])
624+
expect(result.outcomes).toEqual([{ status: "failed", reason: "missing", attempts: 1 }])
625+
}, 15000)
626+
627+
test("permission rejection gets a named corrective turn before an accepted result", async () => {
628+
const result = await run(['{"result":"accepted"}'], 2, { denied: true })
629+
expect(result.requests).toHaveLength(2)
630+
expect(result.requests.map((request) => request.tool_choice)).toEqual([
631+
"required",
632+
{ type: "function", function: { name: "StructuredOutput" } },
633+
])
634+
// The reminder names the denied tool but never the arguments the model sent.
635+
const reminder = JSON.stringify((result.requests[1].messages as unknown[]).at(-1))
636+
expect(reminder).toContain("Permission denied for read. Using what you already have")
637+
expect(reminder).not.toContain("outside-fixture")
638+
expect(result.info.structured).toEqual({ result: "accepted" })
639+
expect(result.rejected).toMatchObject([{ attempt: 1, maxAttempts: 3, errors: [{ keyword: "denied" }] }])
640+
expect(result.outcomes).toEqual([{ status: "accepted", attempts: 2, value: { result: "accepted" } }])
641+
}, 15000)
642+
643+
test("permission rejection without a schema still stops the loop", async () => {
644+
const result = await run(["prose"], 0, { denied: true, plain: true })
645+
expect(result.requests).toHaveLength(1)
646+
expect(result.info.error).toBeUndefined()
647+
expect(result.rejected).toEqual([])
648+
expect(result.outcomes).toEqual([])
621649
}, 15000)
622650

623651
test("diagnostic paths clip every model-chosen property segment without including values", () => {

0 commit comments

Comments
 (0)