Repository navigation
Retain safe ACP diagnostics in saved Prompt failures - #1098
Conversation
PR #1098: Retain safe ACP diagnostics in saved Prompt failures22 files, +1065 / -30 Scope🔴 PR has 1095 lines changed. Split into focused PRs. 🟡 1095 lines changed. PRs under 400 receive more thorough review. 🟡 22 files changed. Are all changes related? 🟡 Changes span 7 directories. StructuralOxlint structural signals:
Slop✅ Slop indicators look low. Static AnalysisOxlint: 27 diagnostics across 5 files (9 rules) no-unused-vars (14): packages/core/src/agent/function-components.ts, packages/cli/src/repl/sidekick/controls.ts CorrectnessNo extraneous code patterns detected. |
taras
left a comment
There was a problem hiding this comment.
Independent review by Claude Fable 5.1 (claude-fable-5-1) of the complete #1039 feature at 58060b03eb4f2eb2e648fd235beacdd9237772c3, against Main 32b53ab9, including the Main integration and CI correction.
Native verdict: pass; no blocking findings. All five recorded acceptance clauses were examined.
Traced the whole feature (648c44a+cbe8b2bf, Main merge 3da4abd, CI repair 58060b0) against Main 32b53ab. Pinned ACPX failRuntimeTurn now projects normalized origin and selected numeric acp code/message/data.details only; raw acp.data, prompts, env and headers are excluded, and the shared/queue turnFailure projection (unreachable from XMD's provider, which only constructs AcpxRuntime/AcpRuntimeManager) is recorded as unchanged. events.ts attaches a non-enumerable diagnostic with the actual turn requestId; cancelled, stop-reason, host-refusal and cleanup AggregateError branches are untouched. Core reads it through a strict zod schema, screens optional text per field then whole with authenticated secretPolicy/scanSecrets (disabled detection respected, failure omits only the diagnostic), and saves it in the existing PromptRecord.error; parsePromptFailure keeps message-only records and refuses malformed present diagnostics. promptFailureFromRecord, REPL model (copied, frozen) and field=error pages expose it; reply/request pages do not. A2 test proves cold reopen with a throwing Agent context and a two-line child prompt count. First negatives for ACP, Core, CLI and runtime-result are retained. Tier AV re-run on exact HEAD: 12/12 pass; checkout clean. No blocking finding; three nonblocking notes.
Nonblocking observations:
-
Minimalism: the property name "executablemd.agent.prompt-diagnostic" is spelled as a separate literal in packages/acp/src/events.ts:172, packages/core/src/agent/errors.ts:10 and three test files. Core's mod.ts already exports AgentPromptError from errors.ts and the ACP package imports core values, so one exported constant would remove the cross-package drift risk. No behavioral consequence today; the ACP and Core tests would both fail if either spelling drifted.
-
packages/core/src/agent/function-components.ts:493 reads the provider Error's diagnostic member (readPromptDiagnostic, including zod traversal of the nested object) before the try block that guards policy and scanning. A provider Error carrying a throwing accessor would therefore propagate out of runPrompt instead of merely losing the diagnostic. This is the same trust level as the existing error.message/name/cause reads at errors.ts:106-113, so it is not a regression, but moving the read inside the try would make the 'failed selection omits only the diagnostic' guarantee total.
-
The retained code "RUNTIME" is the ACPX normalizer's default (ipc chunk resolveOutputErrorCode falls back to options.defaultCode ?? "RUNTIME") and is saved for every in-process failure, including the message-only case through the real runtime (released-runtime.test.ts asserts code: "RUNTIME" alongside a -32041 protocol error). specs/acp-client-spec.md documents it as 'the normalizer's fallback, not a backend diagnosis', which satisfies honest-absence, but a reader of the saved diagnostic must know that this field does not distinguish adapter from backend; the discriminating facts are acp.code/details and origin.
Review bookkeeping: the model call completed, but the delivery response schema rejected its summary (1314 characters; maximum1,200). The original response is preserved; no model retry or correction call was made. That formatting disposition and required exact-head CI remain separate delivery gates.
taras
left a comment
There was a problem hiding this comment.
Independent closeout by Claude Fable 5.1 (claude-fable-5-1) at 9831766bbefe10a356f228f423493783b53bbb16, continuing the original reviewer conversation and reusing its full-feature review at 58060b03. This closeout covers the Journal amendment and final typing fixture correction.
Verdict: pass; no blocking findings. All five recorded acceptance clauses were examined. The joined result passed the delivery response schema.
Reviewed the Journal amendment (0a19aee + one-line typing fixture 9831766) on top of my retained 58060b0 judgment; ACP, Core and vendor code are untouched since then. ReplAgentTurn now carries one detached frozen saved error (nested diagnostic/acp frozen) and the Prompt-page diagnostic member is gone. Journal joins each record to its entry-local turn by the same marker entryRecords already used, exposing promptError only for failed/cancelled turns with a saved error, null otherwise. Cold reopen with a throwing Agent context and an exact child prompt count prove no resend; component, attachment and CLI pages agree. Tier AV re-run on exact HEAD: 12/12. Checkout clean. No blocking finding.
Nonblocking observations:
-
packages/cli/src/repl/sidekick/entry-inspection.ts:338-343 runs entry.turns.find per record on every Journal request, O(records × turns), before yq. model.ts entryRecords (1211) already resolves the same marker join with a Map in O(records + turns), and ReplRecord is built with the turn in hand, so the smaller alternative is either a request-local Map in readEntryJournal or carrying the error on the record. The architect's bounded count (20 records, 9 turns, 144 comparisons) shows repeated work, not a measured latency regression; Journal reads are Sidekick requests, not a keystroke path.
-
Retained acp.details is unbounded: packages/core/src/agent/errors.ts declares details: z.string() with no length cap, so an adapter-controlled string of any size is saved whole in the Prompt record and re-serialized into every Journal query input (entry-inspection.ts:331-345). The new A2 fixture saves a ~1.08 MB details string; with it present, any promptError object and therefore
.records,[.records[] | select(.prompt != null and .status == "failed")](the example in the Journal tool description at controls.ts:663) refuse as oversized, and discovery needs a projection such as del(.promptError). Reply text and exec channels have no bound either, so this matches existing retention policy, and the refusal is clear and paginated details remain readable; whether the issue's 'compact diagnostic' warrants a cap at Core selection is a product choice for the coordinator. The tooLarge() hint at journal-query.ts:70 names only exec channels although repl-spec.md now also names .promptError.diagnostic.acp.details. -
The three earlier nonblocking notes still apply unchanged: the diagnostic property name is a separate literal in packages/acp/src/events.ts:172 and packages/core/src/agent/errors.ts:10; readPromptDiagnostic reads the provider Error's member at function-components.ts:493 before the try that guards screening; code "RUNTIME" is the ACPX normalizer's default and is documented as such, not a backend diagnosis.
Delivery evidence: CI2412 completed successfully on this exact commit, with all 56 jobs complete; the separate automated PR review also passed. Source remains unchanged. No further repair or merge is included in this review.
Why
ACP Prompt failures can share a readable summary while carrying different supported classifications and details. XMD dropped that information before saving the Prompt, making later diagnosis depend on separate runtime logs. Closes #1039.
What changes
Safe available failure details survive in the saved Prompt error. Read them through the same Journal query used for shell-command debugging:
Each record includes
promptError: the selected saved error, or null. Message-only failures retain their message with the diagnostic absent. Existing numeric Prompt identity, marker, status and shell fields keep their meanings.The complete selected entry is queried before pagination. Component, Sidekick attachment and CLI return the same
{value,next,prev,last}shape, with navigation expressed as reusable queries. A matching failed Prompt can be found beyond the first record page. Large selected objects retain the existing refusal; a narrower string selection uses the existing Unicode pages.Existing Prompt reply/request/error text reads keep their prior shape. The earlier branch-only diagnostic member on Prompt text pages is removed.
How it works
The pinned in-process ACPX result retains selected normalization/protocol facts. The terminal Error carries them into Core's existing Prompt failure serializer. Core screens the new optional diagnostic before constructing the saved error, preserving useful safe fields and omitting unsafe or unverifiable optional detail. Original failure fields and the whole-event secret gate keep their behavior.
Core's parser and reconstructed Prompt error preserve the selected diagnostic. The REPL model copies/freezes the complete selected error and nested containers. Journal correlates it by the entry-local record marker. No arbitrary protocol data, provider files, raw prompt payloads, environment or headers are included in the diagnostic projection, and no diagnostic changes failure, refusal, cancellation, cleanup, checkpoint or uncertainty authority. Retryable is descriptive and grants no resend.
Review guide
Review the complete feature at
9831766bbefe10a356f228f423493783b53bbb16against Main32b53ab9, including original feature648c44a3/cbe8b2bf, mechanical Main integration3da4abde, CI correction58060b03, the Journal amendment0a19aeef, and the one-line typing fixture correction9831766b. Reviewing only the latest tests does not cover the original feature.Trace ACPX normalization through
packages/acp/src/events.ts, Core's saved error ownership, the frozen REPL model and Journal query projection. Inspect the existing query/pagination and shared attachment/CLI consumers rather than treating the component result alone as qualification.The prior Claude Fable 5.1 full-feature findings at 58060b03 reported no blockers and three nonblocking observations. Its overlong summary and original result remain preserved and disclosed there. The same original native reviewer completed the affected closeout at 9831766b: pass, no blocking findings, all five acceptance clauses examined, and the response passed the delivery schema. The closeout reviews the Journal amendment and final typing fixture while reusing the earlier full-feature judgment.
How to verify it
cbe8b2bfpassed ACP13 tests/103 steps, Core12/74, controls10/65 and offline vendor verification1/12. The original Architect and Planner passed their frozen full-feature responsibilities there. Main integration preserved the feature patch byte-for-byte; those judgments retain their original provenance.58060b03fixes types without changing the runtime schema, and uses noncredential dummy values in two excluded-data fixtures without changing the scanner or positive credential contrasts. Exact-head CI2410 passed all 56 jobs, and the automated PR review passed on 580.0a19aeef: controls10 groups/66 steps, selected attachment outer suite1/17 steps, vendor1/12 steps. The attachment suite executed the amended shell-and-Prompt component/attachment/CLI parity witness.errormember. Signed child9831766badds exactlyerror: undefinedto that completed-turn fixture. Both failed typechecks (deno task checkand strict Node tsc) and its four-test regression pass on a candidate matched exactly to the commit. Production code, documentation and all Journal behavior are byte-identical to reviewed 0a19aee. The shipped mechanical review policy selects no additional ordinary reviewers; original feedback provenance remains at 0a19aee.Scope
The change covers the pinned in-process result projection/provenance, saved Prompt diagnostics, the existing passive Journal projection, focused regressions and affected documentation. Pristine upstream bytes, dependencies, secret-policy implementation, query/navigation algorithms, test-file discovery, exclusions, runner commands and weights are unchanged.
Risks and limitations
Only supported selected fields are retained, including an actual string at protocol
data.details. Arbitrary adapter data stays excluded. This does not establish the cause of the historical Internal error or whether its backend accepted the request. RUNTIME is the normalizer's fallback, not a backend diagnosis.The Architect and independent Fable closeout retain the request-local repeated marker lookup as a nonblocking cleanup observation. A bounded comparison-count probe establishes extra work; it supplies no timing or responsiveness claim. Fable also notes that retained diagnostic details have no size cap: a large selected error object can hit the existing query-result bound, requiring a narrower projection or the paginated details string. This matches existing retention/refusal behavior; a new retention cap remains a product choice rather than a repair in this PR. The query refusal hint currently names exec channels while the specification also describes Prompt details. Earlier nonblocking observations remain linked in the full-feature review. No additional correction round is implied.
Scope confirmation