Repository navigation
bug(cli): phase uat-passed and verify.artifacts exit 0 while their JSON reports failure #4686
Description
Pre-submission checklist
- I have searched existing issues and this bug has not already been reported
Searched for uat-passed exit code and verify.artifacts exit. #4631 (gates fail closed when they find nothing) is related in spirit but covers scope resolution, not verdict exit codes.
GSD Version
1.13.0
Runtime
Claude Code
Operating System
Linux (Fedora/RHEL)
Node.js Version
v26.8.2
Shell
bash
Installation Method
Runtime-home install at ~/.claude/gsd-core (VERSION reads 1.13.0; the original install method was not recorded)
What happened?
Two gate verbs report a negative verdict only in their JSON and exit 0:
phase uat-passed 47 --require-verificationprinted"passed": falsewith"blockers": ["policy: verification status=stale"]and exited 0. It did the same later with"blockers": ["47-VERIFICATION.md: verification status=human_needed", …].query verify.artifacts <phase-dir>/47-07-PLAN.mdprinted{"all_passed": false, "passed": 1, "total": 2, …}— one artifact failing withMissing pattern: …— and exited 0. Adding--exit-contract=v2(ADR-3889,bin/lib/cli-exit.cjs) also exited 0.
A shell caller that gates on $?, set -e or && proceeds as if both gates passed. The verbs do use non-zero for errors — phase uat-passed on a phase that does not exist exits 1 — so a caller cannot tell "the check ran and failed" from "the check passed" by exit status.
What did you expect?
A negative verdict exits non-zero, at least under --exit-contract=v2, distinct from the exit code used for errors. If exit 0 on a failing verdict is intentional, the verbs' help and the exit-contract ADR should say so, so workflows and scripts know to parse the JSON.
Steps to reproduce
- Pick a plan with a
must_haves.artifactsentry whosecontains:pattern is absent from its file. node ~/.claude/gsd-core/bin/gsd-tools.cjs query verify.artifacts <plan>; echo "exit=$?"→ JSON with"all_passed": false, thenexit=0.- Repeat with
--exit-contract=v2→ stillexit=0. - For a phase whose verification is
staleorhuman_needed:node ~/.claude/gsd-core/bin/gsd-tools.cjs phase uat-passed <N> --require-verification; echo "exit=$?"→"passed": false, thenexit=0.
Error output / logs
{"all_passed":false,"passed":1,"total":2,"failed":[{"path":".planning/user/DEV-SETUP-CHECKLIST.md","issues":["Missing pattern: …"]}]}
exit=0
(failed above is a jq projection of the artifacts array.)
How often does this happen?
Every time (100% reproducible)
Impact
Moderate — Feature is broken but I have a workaround
Workaround (if any)
Gate on the JSON field (jq -r .passed, jq -r .all_passed), never on the exit code.
Additional context
Found in the same session as #4682 and #4683.
Privacy Checklist
- I have reviewed all pasted output for PII (usernames, paths, API keys) and redacted where necessary
Activity
trek-e commentedon Sep 13, 2026
Diagnosis
Reproduced: Yes, by code inspection (control-flow trace, matching the reporter's repro exactly, including
--exit-contract=v2).Root cause: Neither
cmdVerifyArtifacts(src/verify.cts:1395) norcmdPhaseUatPassed(src/phase.cts:4655) ever declares an outcome to the exit-contract seam.output()'s only exit-relevant hook,isPayloadCarriedError(src/io.cts:176), fires solely on an{error: ...}-shaped payload (used by these same functions' precondition arms, e.g. "File not found" — which is why those correctly exit 1 today). A computed domain verdict (all_passed: false/passed: false) is not{error}-shaped, so nothing ever setsrunMain's (src/cli-exit.cts:317) pending-outcome cell, andprocess.exitCodestays at its default0— under v1 and v2, sincegetContractVersion()only changes how a declared outcome projects, and none is ever declared here. This is a defect, not contract-as-designed: ADR-3889's Decision §1 states the0/1band explicitly and unversioned — "0Free. Pass — the operation ran and its verdict is affirmative" / "1Free. Fail — the operation ran and its verdict is negative" — so a negative verdict is supposed to exit 1 today, not only under--exit-contract=v2. The mechanism the ADR designed for this (main()returning'PASS'/'FAIL'forrunMainto project) has simply never been wired into these two verbs — a repo-wide grep forreturn 'FAIL'/return 'PASS'acrosssrc/*.ctsfinds zero call sites at all.Introduced by: Long-standing —
get_timelineoncmdVerifyArtifactsshows the same un-declared-outcome shape across all 21 recorded versions back to 2026-07-30, predating ADR-3889 (2026-08-26) itself.Blast radius: A correctly-scoped fix touches only the two verb functions (declare
'FAIL'/'PASS', or setprocess.exitCodedirectly, based on each function's own computed verdict) and does not require changingoutput(),runMain, or the exit-code registry. The shared seam those two functions would plug into is used by effectively every CLI command (get_impactmeasuredCRITICAL, 180 affected symbols via the sharedoutput()/cli-exithelper fan-out) — real reason to keep the fix narrowly scoped to the two verbs' own finaloutput()call, not the shared seam.Coupled (must not miss): The existing "File not found" / "No must_haves.artifacts" precondition arms in
cmdVerifyArtifactsalready exit 1 correctly via{error}— must not be touched. Any other verb with the same un-declared plain-output()-verdict shape has the identical latent bug, but is out of scope for this issue's exact repro.Regression test:
query verify.artifacts <plan>on a plan with a failingmust_haves.artifactscheck, andphase uat-passed <N> --require-verificationon a phase withstale/human_neededverification, must both exit non-zero (with--exit-contract=v2and without) while still printing their current JSON verdict unchanged; the affirmative-verdict (exit 0) and precondition-error (exit via{error}) paths must remain byte-identical.Agent Brief
Category: bug
Summary: Two CLI gate verbs report a negative verdict only in their JSON payload and always exit 0, so a caller cannot detect failure via
$?.Current behavior: A failing artifact-verification check or a failing UAT-passed policy check prints the failure in JSON (
all_passed: false/passed: falsewith blockers) but the process exits 0 regardless of the exit-contract version selected.Desired behavior: When the computed verdict is negative, the process must exit non-zero (matching the exit-contract's documented
0=pass/1=fail band, unversioned — this must work under the default contract, not only under--exit-contract=v2), while the JSON payload's content and the existing precondition-error exit codes (file/phase not found, etc.) stay exactly as they are today.Key interfaces:
phase uat-passed --require-verification,query verify.artifacts, the exit-contract declaration mechanism (a command declaringPASS/FAILfor the process exit code),--exit-contract=v2/GSD_EXIT_CONTRACT.Acceptance criteria:
query verify.artifactson a plan with a failing artifact check exits non-zero (both with and without--exit-contract=v2) while still printingall_passed: falsein its JSON.phase uat-passed --require-verificationon a phase with astale/human_needed/policy-blocked verification status exits non-zero while still printingpassed: falseandblockersin its JSON. Both verbs continue to exit 0 on an affirmative verdict. Both verbs' existing precondition-error arms (missing plan/phase/file) are unchanged.Out of scope: Changing the shared
output()/runMain/exit-code-registry seam itself; auditing or fixing other verbs that may share this same latent pattern; changing the JSON payload shape of either verb.chore(open-gsd#4686): backfill changeset pr number to 4938
chore(workspace): sync task 4686 planning artifacts and update person…
fix(open-gsd#4686): exit 1 on negative verdict in phase uat-passed an…
chore(open-gsd#4686): backfill changeset pr number to 4938
fix(open-gsd#4686): address review feedback with dedicated VERDICT_FA…
trek-e commentedon Sep 27, 2026
Closing as superseded by epic #5056, not as won't-fix. This defect will be architected away in Phase 8: Phase 8 derives every gate verb's exit code from its typed verdict through the process exit contract (
cli-exit), so no verb chooses its own exit status (ADR-5057 §4).The point-fix PR #4938 was closed unmerged for the same reason, with thanks. The Phase 8 PR will reference this issue with
Refs #4686and keep a fail-first regression for it. The mapping is recorded in Phase 0 (#5057, ADR-5057).docs(quick): record retirement of fix(open-gsd#4686) superseded by up…
ci(#5170): the gate-evidence drift guard is path-sensitive, per-symbo…
ci(#5170): the gate-evidence drift guard is path-sensitive, per-symbo…
refactor(#5170): typed gate evidence and a verdict-driven exit status…