Skip to content

Retain safe ACP diagnostics in saved Prompt failures - #1098

Merged
taras merged 6 commits into
mainfrom
codex/acp-error-details-1039
Nov 4, 2026
Merged

taras merged 6 commits into
mainfrom
codex/acp-error-details-1039

Conversation

@taras

@taras taras commented Nov 4, 2026 •

Copy link
Copy Markdown
Owner

Why

ACP Prompt failures can share a readable summary while carrying different supported classifications and details. XMD dropped that information before saving the Prompt, making later diagnosis depend on separate runtime logs. Closes #1039.

What changes

Safe available failure details survive in the saved Prompt error. Read them through the same Journal query used for shell-command debugging:

<REPL.Entry.Journal entry={1}
  yq={'[.records[] | select(.prompt != null and .status == "failed")]'} />

Each record includes promptError: the selected saved error, or null. Message-only failures retain their message with the diagnostic absent. Existing numeric Prompt identity, marker, status and shell fields keep their meanings.

The complete selected entry is queried before pagination. Component, Sidekick attachment and CLI return the same {value,next,prev,last} shape, with navigation expressed as reusable queries. A matching failed Prompt can be found beyond the first record page. Large selected objects retain the existing refusal; a narrower string selection uses the existing Unicode pages.

Existing Prompt reply/request/error text reads keep their prior shape. The earlier branch-only diagnostic member on Prompt text pages is removed.

How it works

The pinned in-process ACPX result retains selected normalization/protocol facts. The terminal Error carries them into Core's existing Prompt failure serializer. Core screens the new optional diagnostic before constructing the saved error, preserving useful safe fields and omitting unsafe or unverifiable optional detail. Original failure fields and the whole-event secret gate keep their behavior.

Core's parser and reconstructed Prompt error preserve the selected diagnostic. The REPL model copies/freezes the complete selected error and nested containers. Journal correlates it by the entry-local record marker. No arbitrary protocol data, provider files, raw prompt payloads, environment or headers are included in the diagnostic projection, and no diagnostic changes failure, refusal, cancellation, cleanup, checkpoint or uncertainty authority. Retryable is descriptive and grants no resend.

Review guide

Review the complete feature at 9831766bbefe10a356f228f423493783b53bbb16 against Main 32b53ab9, including original feature 648c44a3/cbe8b2bf, mechanical Main integration 3da4abde, CI correction 58060b03, the Journal amendment 0a19aeef, and the one-line typing fixture correction 9831766b. Reviewing only the latest tests does not cover the original feature.

Trace ACPX normalization through packages/acp/src/events.ts, Core's saved error ownership, the frozen REPL model and Journal query projection. Inspect the existing query/pagination and shared attachment/CLI consumers rather than treating the component result alone as qualification.

The prior Claude Fable 5.1 full-feature findings at 58060b03 reported no blockers and three nonblocking observations. Its overlong summary and original result remain preserved and disclosed there. The same original native reviewer completed the affected closeout at 9831766b: pass, no blocking findings, all five acceptance clauses examined, and the response passed the delivery schema. The closeout reviews the Journal amendment and final typing fixture while reusing the earlier full-feature judgment.

How to verify it

  • Original production paths fail the retained witnesses for lost runtime, terminal, saved and inspection details. First failures and exact candidates are preserved.
  • Original feedback at signed cbe8b2bf passed ACP13 tests/103 steps, Core12/74, controls10/65 and offline vendor verification1/12. The original Architect and Planner passed their frozen full-feature responsibilities there. Main integration preserved the feature patch byte-for-byte; those judgments retain their original provenance.
  • CI2409 exposed strict Node event-array typing and an explicit JSR schema annotation. The same JSR issue failed the staged consumer. 58060b03 fixes types without changing the runtime schema, and uses noncredential dummy values in two excluded-data fixtures without changing the scanner or positive credential contrasts. Exact-head CI2410 passed all 56 jobs, and the automated PR review passed on 580.
  • The requested Journal amendment has three passing candidates whose complete binary patches match signed 0a19aeef: controls10 groups/66 steps, selected attachment outer suite1/17 steps, vendor1/12 steps. The attachment suite executed the amended shell-and-Prompt component/attachment/CLI parity witness.
  • The production save/cold-read journey distinguishes two actual equal-summary failures, preserves their classifications/correlation, locates failures after record 20, traverses narrower Unicode detail pages and leaves provider count exactly two, history bytes unchanged and entry count unchanged. It covers oversized object/item refusal, null/message-only/frozen error distinctions, raw-payload exclusions and unchanged Prompt summary pages.
  • The original Architect passed affected S4-J/S5-J on exact 0a; the original Planner passed the frozen affected acceptance matrix on exact 0a. Neither repeated passing suites or waited for CI. Shipped Phase.Accept/Ticket.Advance computed exact-head feedback readiness from these actual saved outcomes with zero provider calls. Prior feedback and failures remain preserved.
  • The complete current feature diff has zero findings under the unchanged secret scanner.
  • CI2411 found one existing typing fixture missing the required error member. Signed child 9831766b adds exactly error: undefined to that completed-turn fixture. Both failed typechecks (deno task check and strict Node tsc) and its four-test regression pass on a candidate matched exactly to the commit. Production code, documentation and all Journal behavior are byte-identical to reviewed 0a19aee. The shipped mechanical review policy selects no additional ordinary reviewers; original feedback provenance remains at 0a19aee.
  • Exact-head CI2412 completed successfully at 9831766: all 56 jobs complete and all 57 check runs accounted for, including the successful separate automated PR review1895. Current Main32b53ab9 is independently proven by its authoritative push CI2408.
  • The same original Claude Fable 5.1 reviewer completed the affected closeout at exact 9831766. Its joined response passed the actual delivery schema, all five clauses were examined, and no blocking finding was reported. The reviewer reran the 12 vendor checks and reported a clean checkout. Root verified the original native identity, canonical response/receipt correspondence and unchanged signed source.

Scope

The change covers the pinned in-process result projection/provenance, saved Prompt diagnostics, the existing passive Journal projection, focused regressions and affected documentation. Pristine upstream bytes, dependencies, secret-policy implementation, query/navigation algorithms, test-file discovery, exclusions, runner commands and weights are unchanged.

Risks and limitations

Only supported selected fields are retained, including an actual string at protocol data.details. Arbitrary adapter data stays excluded. This does not establish the cause of the historical Internal error or whether its backend accepted the request. RUNTIME is the normalizer's fallback, not a backend diagnosis.

The Architect and independent Fable closeout retain the request-local repeated marker lookup as a nonblocking cleanup observation. A bounded comparison-count probe establishes extra work; it supplies no timing or responsiveness claim. Fable also notes that retained diagnostic details have no size cap: a large selected error object can hit the existing query-result bound, requiring a narrower projection or the paginated details string. This matches existing retention/refusal behavior; a new retention cap remains a product choice rather than a repair in this PR. The query refusal hint currently names exec channels while the specification also describes Prompt details. Earlier nonblocking observations remain linked in the full-feature review. No additional correction round is implied.

Scope confirmation

  • Every changed file supports preserved ACP failure evidence or the requested Journal inspection path.
  • Generated changes and provenance are identified.
  • No unrelated cleanup, dependency change or live-provider qualification is included.
  • The description distinguishes exact-head feedback, historical reviews, completed independent closeout and exact-head delivery checks.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

PR #1098: Retain safe ACP diagnostics in saved Prompt failures

22 files, +1065 / -30

Scope

🔴 PR has 1095 lines changed. Split into focused PRs.

🟡 1095 lines changed. PRs under 400 receive more thorough review.

🟡 22 files changed. Are all changes related?

🟡 Changes span 7 directories.

Structural

Oxlint structural signals:

  • no-unused-vars ×14: packages/core/src/agent/function-components.ts, packages/cli/src/repl/sidekick/controls.ts
  • no-unnecessary-type-assertion ×2: packages/cli/src/repl/model.ts

Slop

✅ Slop indicators look low.

Static Analysis

Oxlint: 27 diagnostics across 5 files (9 rules)
Density: 0.025 violations/added-line

no-unused-vars (14): packages/core/src/agent/function-components.ts, packages/cli/src/repl/sidekick/controls.ts
no-unnecessary-type-conversion (3): packages/cli/src/repl/sidekick/controls.ts, packages/core/src/agent/function-components.ts, packages/cli/src/repl/model.ts
no-shadow (2): packages/core/src/agent/journal.ts, packages/core/src/agent/function-components.ts
no-misused-spread (2): packages/cli/src/repl/sidekick/controls.ts, packages/cli/src/repl/sidekick/entry-inspection.ts
no-unnecessary-type-assertion (2): packages/cli/src/repl/model.ts
no-array-sort (1): packages/cli/src/repl/model.ts
no-base-to-string (1): packages/core/src/agent/function-components.ts
unbound-method (1): packages/core/src/agent/function-components.ts
no-unnecessary-template-expression (1): packages/cli/src/repl/model.ts

Correctness

No extraneous code patterns detected.

@taras taras left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review by Claude Fable 5.1 (claude-fable-5-1) of the complete #1039 feature at 58060b03eb4f2eb2e648fd235beacdd9237772c3, against Main 32b53ab9, including the Main integration and CI correction.

Native verdict: pass; no blocking findings. All five recorded acceptance clauses were examined.

Traced the whole feature (648c44a+cbe8b2bf, Main merge 3da4abd, CI repair 58060b0) against Main 32b53ab. Pinned ACPX failRuntimeTurn now projects normalized origin and selected numeric acp code/message/data.details only; raw acp.data, prompts, env and headers are excluded, and the shared/queue turnFailure projection (unreachable from XMD's provider, which only constructs AcpxRuntime/AcpRuntimeManager) is recorded as unchanged. events.ts attaches a non-enumerable diagnostic with the actual turn requestId; cancelled, stop-reason, host-refusal and cleanup AggregateError branches are untouched. Core reads it through a strict zod schema, screens optional text per field then whole with authenticated secretPolicy/scanSecrets (disabled detection respected, failure omits only the diagnostic), and saves it in the existing PromptRecord.error; parsePromptFailure keeps message-only records and refuses malformed present diagnostics. promptFailureFromRecord, REPL model (copied, frozen) and field=error pages expose it; reply/request pages do not. A2 test proves cold reopen with a throwing Agent context and a two-line child prompt count. First negatives for ACP, Core, CLI and runtime-result are retained. Tier AV re-run on exact HEAD: 12/12 pass; checkout clean. No blocking finding; three nonblocking notes.

Nonblocking observations:

  1. Minimalism: the property name "executablemd.agent.prompt-diagnostic" is spelled as a separate literal in packages/acp/src/events.ts:172, packages/core/src/agent/errors.ts:10 and three test files. Core's mod.ts already exports AgentPromptError from errors.ts and the ACP package imports core values, so one exported constant would remove the cross-package drift risk. No behavioral consequence today; the ACP and Core tests would both fail if either spelling drifted.

  2. packages/core/src/agent/function-components.ts:493 reads the provider Error's diagnostic member (readPromptDiagnostic, including zod traversal of the nested object) before the try block that guards policy and scanning. A provider Error carrying a throwing accessor would therefore propagate out of runPrompt instead of merely losing the diagnostic. This is the same trust level as the existing error.message/name/cause reads at errors.ts:106-113, so it is not a regression, but moving the read inside the try would make the 'failed selection omits only the diagnostic' guarantee total.

  3. The retained code "RUNTIME" is the ACPX normalizer's default (ipc chunk resolveOutputErrorCode falls back to options.defaultCode ?? "RUNTIME") and is saved for every in-process failure, including the message-only case through the real runtime (released-runtime.test.ts asserts code: "RUNTIME" alongside a -32041 protocol error). specs/acp-client-spec.md documents it as 'the normalizer's fallback, not a backend diagnosis', which satisfies honest-absence, but a reader of the saved diagnostic must know that this field does not distinguish adapter from backend; the discriminating facts are acp.code/details and origin.

Review bookkeeping: the model call completed, but the delivery response schema rejected its summary (1314 characters; maximum1,200). The original response is preserved; no model retry or correction call was made. That formatting disposition and required exact-head CI remain separate delivery gates.

@taras
taras marked this pull request as ready for review November 4, 2026 03:42

@taras taras left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent closeout by Claude Fable 5.1 (claude-fable-5-1) at 9831766bbefe10a356f228f423493783b53bbb16, continuing the original reviewer conversation and reusing its full-feature review at 58060b03. This closeout covers the Journal amendment and final typing fixture correction.

Verdict: pass; no blocking findings. All five recorded acceptance clauses were examined. The joined result passed the delivery response schema.

Reviewed the Journal amendment (0a19aee + one-line typing fixture 9831766) on top of my retained 58060b0 judgment; ACP, Core and vendor code are untouched since then. ReplAgentTurn now carries one detached frozen saved error (nested diagnostic/acp frozen) and the Prompt-page diagnostic member is gone. Journal joins each record to its entry-local turn by the same marker entryRecords already used, exposing promptError only for failed/cancelled turns with a saved error, null otherwise. Cold reopen with a throwing Agent context and an exact child prompt count prove no resend; component, attachment and CLI pages agree. Tier AV re-run on exact HEAD: 12/12. Checkout clean. No blocking finding.

Nonblocking observations:

  1. packages/cli/src/repl/sidekick/entry-inspection.ts:338-343 runs entry.turns.find per record on every Journal request, O(records × turns), before yq. model.ts entryRecords (1211) already resolves the same marker join with a Map in O(records + turns), and ReplRecord is built with the turn in hand, so the smaller alternative is either a request-local Map in readEntryJournal or carrying the error on the record. The architect's bounded count (20 records, 9 turns, 144 comparisons) shows repeated work, not a measured latency regression; Journal reads are Sidekick requests, not a keystroke path.

  2. Retained acp.details is unbounded: packages/core/src/agent/errors.ts declares details: z.string() with no length cap, so an adapter-controlled string of any size is saved whole in the Prompt record and re-serialized into every Journal query input (entry-inspection.ts:331-345). The new A2 fixture saves a ~1.08 MB details string; with it present, any promptError object and therefore .records, [.records[] | select(.prompt != null and .status == "failed")] (the example in the Journal tool description at controls.ts:663) refuse as oversized, and discovery needs a projection such as del(.promptError). Reply text and exec channels have no bound either, so this matches existing retention policy, and the refusal is clear and paginated details remain readable; whether the issue's 'compact diagnostic' warrants a cap at Core selection is a product choice for the coordinator. The tooLarge() hint at journal-query.ts:70 names only exec channels although repl-spec.md now also names .promptError.diagnostic.acp.details.

  3. The three earlier nonblocking notes still apply unchanged: the diagnostic property name is a separate literal in packages/acp/src/events.ts:172 and packages/core/src/agent/errors.ts:10; readPromptDiagnostic reads the provider Error's member at function-components.ts:493 before the try that guards screening; code "RUNTIME" is the ACPX normalizer's default and is documented as such, not a backend diagnosis.

Delivery evidence: CI2412 completed successfully on this exact commit, with all 56 jobs complete; the separate automated PR review also passed. Source remains unchanged. No further repair or merge is included in this review.

@taras
taras merged commit 25368ab into main Nov 4, 2026
58 checks passed
@taras
taras deleted the codex/acp-error-details-1039 branch November 4, 2026 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preserve ACP error details in saved Prompt failures

1 participant