You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 94743b5
Browse filesBrowse the repository at this point in the historyBrowse files
fix(evidence): address the PR review — orthogonal effects, frozen schemas, and reviewer-safe prose (#409)
Eight review comments. Five affected gate correctness or wire compatibility.
**Effects are not totally ordered by obligation.** Declaring `financial_write`
over an inferred `external_communication` outranks it on risk, but requires no
confirmation — which is exactly what communicating outward requires. Reading
rank alone made that action pass-eligible with no gap and no
external-communication finding while the external-write risk tag sat untouched
in the same report. `declaration_covers` now requires the declaration to rank at
or above the observation under **both** published rank orders *and* to oblige at
least that observation's built-in controls. The obligations move into
`BUILTIN_EFFECT_OBLIGATIONS`, pinned against the branches they mirror by a test
that walks each entry through a real scan.
Coverage also reads the whole reviewed surface rather than the `effect` field
alone: a `risk_tags: [financial_action]` entry produces a policy-eligible
`financial_write` claim and applies the financial-write controls, so a heuristic
saying the same thing is already accounted for. The shipped
`ai_generated_refund_pr` fixture declares exactly that pair and found this.
**Four published schemas were mutated in place.** The new enum value propagated
into packet v0.12, verifier v0.9, capability-lock v0.6, and capability-lock-diff
v0.7, all keeping their version identifiers — so a consumer pinned to any of
them rejected artifacts that document was supposed to describe.
`generate_schemas.py --check` proves committed == generated and cannot catch it.
Those four are restored byte-for-byte and the versions advance to 0.13, 0.10,
0.7, and 0.8. Prior artifacts still read: packet gains a v0.12 upgrade arm, the
verifier's legacy reader gains 0.9, and a v0.6 lock is advanced on load instead
of being rejected, which the bump would otherwise have done to every committed
`capabilities.lock.json`. A parametrised test now asserts the frozen documents
match `origin/main` byte-for-byte and that emitted artifacts validate against
their own current schema.
**The two comparators disagreed.** `EFFECT_EVIDENCE_RANK` orders `write` below
`privileged_data_access` and `ACTION_EFFECT_RANK` the other way, so a declared
`privileged_data_access` read as covered here while the policy path raised
`mixed_policy_evidence` — a verdict no override could reach, contradicting the
documented promise. Both consumers now call `declaration_covers`, which requires
both orders to agree; nothing that gated before stops gating.
**Two documented policy paths did not exist.** `ActionFact` has no `override`
projection, so `require: {override.reason: ...}` resolved to `unknown_path` and
produced a blocking false positive. Both rows are removed and the docs say where
a recorded override *is* readable.
**A visually blank reason satisfied the reviewed-reason requirement.**
`str.strip()` accepts U+200B. The reason now requires visible content and
rejects invisible and direction-altering code points; `has_visible_content` moved
to `schemas/_text.py` so a manifest validator can reach it without `schemas`
importing `core`.
Reviewer surface: the reason is folded through `one_line` before truncation, so
a `\n- Agent may merge: true` cannot render as a separate authority-looking
line; and each observation renders beside the producers that actually made it
rather than the strongest effect beside a union of every source.
Also: `benchmark/safety-qualification/README.md` required report schema 0.35
while the runner now requires 0.36, so following the checked-in procedure
produced receipts the runner rejects.
Co-Authored-By: Claude Opus 5 <[email protected]>
Copy file name to clipboardExpand all lines: README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -669,7 +669,7 @@ artifacts — in read order:
669
669
- **`agents-shipgate-reports/capabilities.lock.json`** + **`agents-shipgate-reports/base.capabilities.lock.json`** + **`agents-shipgate-reports/capability-lock-diff.{json,md}`** — the **capability review primitive**. Verify always emits the head lock after a successful scan; it emits the base lock and diff when the base scan can be materialized, falling back to the reviewed committed lock at `.agents-shipgate/capabilities.lock.json` if needed.
670
670
- **Gate source of truth** — `report.json.release_decision.decision` (`passed | review_required | insufficient_evidence | blocked`). `merge_verdict` is a deterministic projection of it; the report stays the one decision engine.
671
671
- **Tool-Use Readiness Report** (supporting) — `agents-shipgate-reports/report.{md,json,sarif}`. Markdown for human release review, JSON for tools and coding agents, SARIF for GitHub code-scanning workflows. This is the underlying check domain the verdict summarizes.
672
-
- **Release Evidence Packet** (supporting) — `agents-shipgate-reports/packet.{md,json,html}` (and `packet.pdf` with the `[pdf]` extras). Reviewer-shaped synthesis with fixed sections, including binding and semantic evidence coverage, the compact evidence matrix, and tool/action diffs when available. Packet outputs are locally redacted by default; see [STABILITY.md §Release Evidence Packet](STABILITY.md#release-evidence-packet-v012).
672
+
- **Release Evidence Packet** (supporting) — `agents-shipgate-reports/packet.{md,json,html}` (and `packet.pdf` with the `[pdf]` extras). Reviewer-shaped synthesis with fixed sections, including binding and semantic evidence coverage, the compact evidence matrix, and tool/action diffs when available. Packet outputs are locally redacted by default; see [STABILITY.md §Release Evidence Packet](STABILITY.md#release-evidence-packet-v013).
0 commit comments