You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2f75964
Browse filesBrowse the repository at this point in the historyBrowse files
fix(skills): pr-review-merge requires the mandated actual-run evidence
review_gate.py ignored target_validation, so a PR changing playbooks,
specs or contracts could merge with no actual-run record. The gate now
reads .github/pr-evidence.json at the PR's base SHA (GitHub contents
API, never the head) and checks target_validation with issue-to-pr's
repo_evidence.py, loaded by path; the evidence-only delta comes from
GitHub compare. Missing, malformed, blocked, uncovered or unverifiable
records are WAITING; DRAFT still comes only from the reviewer's P1.
The manifest gains required_paths (playbooks, verification specs,
contracts), for which not-needed is not accepted. GitHub answers in
the tests are read-only gh api captures from kjelly/pilot. Integration
tests lock a per-test PRM_STATE_DIR instead of ~/.cache.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copy file name to clipboardExpand all lines: .agents/skills/pr-review-merge/PROJECT-CHANGES.md
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,14 +13,15 @@ below, then run `make skill-test` and `go test ./cmd/pilot/cmd -run TestAgentSki
13
13
|`SKILL.md`| "Project runtime configuration" and "Pilot: repository rules the review and the merge must respect" sections (merge commits, no branch rules, history reference, mandated actual-run evidence as a P1, cost, serial merges); merge-policy sentence in §1; `--method merge` in the §4 example |
14
14
|`platforms/{claude,codex,agy}/agents/prm-{reviewer,dependency}.*`| one prompt body per role on every platform (upstream shipped three different texts); pilot lines (history reference, `target_validation`, missing actual-run record is a blocking P1, test scope); Claude `tools`/`model`/`effort` and Codex `model`/`model_reasoning_effort` pins |
15
15
|`scripts/apply_decision.py`|`--method` defaults to `merge` (upstream: `squash`); locked by `tests/test_project_defaults.py`|
16
-
|`references/evidence-contract.md`|`target_validation` object in the example and its rules |
|`references/evidence-contract.md`|`target_validation` object in the example and its rules (enforced by the gate, see the last row) |
17
+
|`references/quality-gate.md`|WAITING row for a missing mandated actual-run record (gate) and DRAFT row for the reviewer's P1 on it; merge command/method wording |
18
18
|`references/workflow.md`| step 3 (history reference + `.github/pr-evidence.json` check), renumbered steps, `Target validation` line in the reviewer return |
19
19
|`references/platforms.md`| rewritten for the repository-local layout (no installer, no global paths) |
20
20
|`tests/test_project_defaults.py`| new |
21
21
|`scripts/install.sh`, `tests/test_installer.py`| removed: the skill is repository-local, nothing installs it elsewhere |
22
22
|`README.md`| install section replaced by the repository layout; examples target `kjelly/pilot`; default merge method wording |
23
23
|`scripts/review_gate.py`, `tests/test_gate.py`, `tests/test_pr_ready_handoff.py` (new), `SKILL.md` ("Hand-off with `pr-ready`", "Disputed findings", §2, §3 table), `references/{evidence-contract,quality-gate,workflow}.md`, `platforms/{claude,codex,agy}/agents/prm-reviewer.*`| no Draft/Ready loop with `pr-ready`: `CHANGES_REQUESTED` and reviewer-confirmed `unrelated_failing_checks` decide `WAITING` instead of `DRAFT` (upstream: `DRAFT`); a finding that repeats a disputed claim carries `disputed_reply_url` and is `DRAFT` only with `new_evidence`, otherwise `WAITING` for a person; severity `P4` accepted (shared rubric); the hand-off test runs both gates over the same GitHub states |
24
+
|`scripts/review_gate.py`, `scripts/gh_runtime.py`, `tests/test_target_validation.py` (new), `tests/fixtures/github_api.json` (new), `tests/{test_gate,test_integration,test_pr_ready_handoff}.py`, `SKILL.md` ("Mandated actual-run evidence", hand-off, §3 table), `README.md`, `references/{evidence-contract,quality-gate,workflow}.md`, `platforms/{claude,codex,agy}/agents/prm-reviewer.*`| the gate enforces the actual-run record of the base branch's `.github/pr-evidence.json`: manifest read at the base SHA via the GitHub contents API, `target_validation` checked with issue-to-pr's `scripts/repo_evidence.py` (loaded by path; `RemoteContext` takes the evidence-only delta from GitHub `compare`), every gap `WAITING`; the manifest's `required_paths` (new key, only this gate reads it) refuse `not-needed`; integration tests use a per-test `PRM_STATE_DIR`|
24
25
25
26
Files not listed here are believed unchanged from upstream 1.0.0; verify against the upstream
Copy file name to clipboardExpand all lines: .agents/skills/pr-review-merge/SKILL.md
+16-9Lines changed: 16 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,20 +51,27 @@ definitions shipped beside it; there is no personal installation. Codex invocati
51
51
-**Mandated actual-run evidence.**`.github/pr-evidence.json` on the base branch
52
52
declares `docs/evidence/**` as evidence-only and requires an `actual-run` record
53
53
for any change to playbooks, specs, contracts, inventory handling or deploy
54
-
orchestration (AGENTS.md §1.4/§1.5). A PR that needs one and has no
55
-
`docs/evidence/<feature>/<date>-<tested-revision>.md` covering the reviewed HEAD
56
-
(or an evidence-only delta from the tested revision) carries a blocking `P1`
57
-
finding at the changed playbook/spec path: that is a confirmed violation of a hard
58
-
repository rule, not a `WAITING` gap. Docs-, skill- or pure-Go changes proven by
59
-
native-tool tests need no VM run; the reviewer says so in `target_validation`.
54
+
orchestration (AGENTS.md §1.4/§1.5). `review_gate.py` enforces it: it reads the
55
+
manifest at the PR's base SHA through the GitHub API (never the head, so a PR cannot
56
+
weaken its own requirement) and checks `target_validation` with issue-to-pr's
57
+
`repo_evidence.py` rules. Missing or malformed, `blocked`, `not-needed` on a PR that
58
+
changes the manifest's `required_paths` (`playbooks/**`, `docs/verification/**`,
59
+
`contracts/**`), or `required` without a committed `docs/evidence/**` record that names
60
+
a tested revision covering HEAD (same SHA, or an ancestor whose delta to HEAD is
61
+
evidence-only) is `WAITING`, never `MERGE` (`references/evidence-contract.md`). The
62
+
reviewer still files a blocking `P1` at the changed path when a change needs a record
63
+
and has none: that is a confirmed violation of a hard repository rule and the only way
64
+
this becomes `DRAFT`. Docs-, skill- or pure-Go changes proven by native-tool tests need
65
+
no VM run; the reviewer says so in `target_validation` with the reason.
60
66
-**Cost.** The full Go suite takes about 45 minutes in CI. Reviewers run the
61
67
packages the diff touches (and `make skill-test` for `.agents/skills` changes)
62
68
locally and rely on the `Go test` check for the whole suite.
63
69
-**Serial merges.** After each merge every remaining PR's base moves; re-fetch
64
70
base/HEAD and arrange a new independent review before acting on the next one.
65
71
-**Hand-off with `pr-ready`.**`pr-ready` marks a PR Ready only when every review
66
72
thread is resolved; it may leave a reviewer-confirmed unrelated failing check, a
67
-
`CHANGES_REQUESTED` review awaiting re-review, and findings it disputed on the PR.
73
+
`CHANGES_REQUESTED` review awaiting re-review, findings it disputed on the PR, and
74
+
(its `pr_gate.py` does not read `.github/pr-evidence.json`) no actual-run record.
68
75
None of these is a reason to return the PR to Draft (that only starts another
69
76
round): the gate decides `WAITING` for them and never `MERGE`.
70
77
`tests/test_pr_ready_handoff.py` runs both gates over the same GitHub states and
@@ -95,9 +102,9 @@ definitions shipped beside it; there is no personal installation. Codex invocati
95
102
96
103
| Decision | Conditions | GitHub action |
97
104
|---|---|---|
98
-
|`MERGE`| Valid independent reviewed HEAD/Base; zero P0–P2 blockers; relevant HEAD-SHA-bound tests succeed; CI all passed (or narrowly justified no-CI policy); required approvals and rules satisfied; no unresolved threads/dependencies; current mergeable and safe |`gh pr merge --match-head-commit <SHA>`; verify `MERGED`|
105
+
|`MERGE`| Valid independent reviewed HEAD/Base; zero P0–P2 blockers; relevant HEAD-SHA-bound tests succeed; the base manifest's required record covers HEAD (or is reasoned `not-needed`); CI all passed (or narrowly justified no-CI policy); required approvals and rules satisfied; no unresolved threads/dependencies; current mergeable and safe |`gh pr merge --match-head-commit <SHA>`; verify `MERGED`|
99
106
|`DRAFT`| Confirmed blocking defect, failed CI not confirmed unrelated, failing tests, substantive review finding (a repeated disputed one only with new evidence), merge conflict or unresolved review thread | Post actionable findings and `gh pr ready --undo`|
100
-
|`WAITING`| Pending CI, pending required approval, `CHANGES_REQUESTED` awaiting re-review, a failing check the reviewer confirmed unrelated to the PR, a disputed finding repeated without new evidence, unknown GitHub status, evidence incomplete, HEAD/Base changed, merge queue, missing access or unmet dependency | Leave Ready **unchanged**; report blocker and recommended next action |
107
+
|`WAITING`| Pending CI, pending required approval, `CHANGES_REQUESTED` awaiting re-review, a failing check the reviewer confirmed unrelated to the PR, a disputed finding repeated without new evidence, the mandated actual-run record missing or not covering HEAD (no reviewer finding), unknown GitHub status, evidence incomplete, HEAD/Base changed, merge queue, missing access or unmet dependency | Leave Ready **unchanged**; report blocker and recommended next action |
101
108
|`SKIP`| Already Draft, closed or merged | No action |
102
109
103
110
`WAITING` is intentional: lack of evidence or a transient pending check is not proof the code is defective. **Do not automatically downgrade in these situations**. If the PR has *any confirmed* blocking defect, classify `DRAFT` even if some checks are pending.
Copy file name to clipboardExpand all lines: .agents/skills/pr-review-merge/platforms/agy/agents/prm-reviewer.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ Read the Root-supplied pilot_history_checks reference, the finding ledger and th
16
16
Read the full diff against the current base, the affected source and shared includes, the linked Issue and acceptance criteria, review comments, CI and tests.
17
17
Root gives you each earlier finding that was disputed on the PR with the reply disputing it; judge each claim yourself from the code. Repeat a disputed claim only with evidence the reply does not answer, and then set disputed_reply_url to the reply and new_evidence to that evidence. A failing CI check goes in unrelated_failing_checks only with evidence that it fails the same way without this diff (for example on the base); that never makes the PR mergeable.
18
18
Assess requirements, correctness, architecture, security, concurrency, edge cases, semantic regression, migration/rollback risk and test adequacy; distinguish executed tests, skipped tests and structural checks.
19
-
Record target_validation = required | not-needed | blockedwith the invariant it rests on. A change that needs a candidate-bound actual-run record (playbooks, specs, contracts, inventory handling, deploy orchestration) and has none covering the reviewed HEAD or an evidence-only delta is a blocking P1 finding at the changed path, not a verbal reservation.
19
+
Record target_validation: decision required | not-needed | blocked, a reason of 30+ characters with the invariant it rests on, and for required the committed docs/evidence artifact and the full 40-hex tested_revision that artifact names; the gate checks these against the base branch's manifest and never accepts not-needed for its required_paths. A change that needs a candidate-bound actual-run record (playbooks, specs, contracts, inventory handling, deploy orchestration) and has none covering the reviewed HEAD or an evidence-only delta is a blocking P1 finding at the changed path, not a verbal reservation.
20
20
Use safe read-only commands and trusted test commands in an isolated worktree, scoped to the packages the diff touches; never modify source, branches or PR state, never comment, approve or merge.
21
21
Return the compact structure of references/evidence-contract.md: exact headRefOid and baseRefOid, verdict PASS | BLOCK | NEEDS_MORE_EVIDENCE, P0–P4 findings with path:line, evidence, reproduction and remediation (origin judged against the PR base per the shared severity rubric), explicit coverage, test results with links, dependency status, target_validation and scope.
22
22
Never invent test success or call a partial diff a complete review; raw diffs and logs stay in separate artifacts. Root owns decisions and GitHub state.
Copy file name to clipboardExpand all lines: .agents/skills/pr-review-merge/platforms/claude/agents/prm-reviewer.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ Read the Root-supplied pilot_history_checks reference, the finding ledger and th
11
11
Read the full diff against the current base, the affected source and shared includes, the linked Issue and acceptance criteria, review comments, CI and tests.
12
12
Root gives you each earlier finding that was disputed on the PR with the reply disputing it; judge each claim yourself from the code. Repeat a disputed claim only with evidence the reply does not answer, and then set disputed_reply_url to the reply and new_evidence to that evidence. A failing CI check goes in unrelated_failing_checks only with evidence that it fails the same way without this diff (for example on the base); that never makes the PR mergeable.
13
13
Assess requirements, correctness, architecture, security, concurrency, edge cases, semantic regression, migration/rollback risk and test adequacy; distinguish executed tests, skipped tests and structural checks.
14
-
Record target_validation = required | not-needed | blockedwith the invariant it rests on. A change that needs a candidate-bound actual-run record (playbooks, specs, contracts, inventory handling, deploy orchestration) and has none covering the reviewed HEAD or an evidence-only delta is a blocking P1 finding at the changed path, not a verbal reservation.
14
+
Record target_validation: decision required | not-needed | blocked, a reason of 30+ characters with the invariant it rests on, and for required the committed docs/evidence artifact and the full 40-hex tested_revision that artifact names; the gate checks these against the base branch's manifest and never accepts not-needed for its required_paths. A change that needs a candidate-bound actual-run record (playbooks, specs, contracts, inventory handling, deploy orchestration) and has none covering the reviewed HEAD or an evidence-only delta is a blocking P1 finding at the changed path, not a verbal reservation.
15
15
Use safe read-only commands and trusted test commands in an isolated worktree, scoped to the packages the diff touches; never modify source, branches or PR state, never comment, approve or merge.
16
16
Return the compact structure of references/evidence-contract.md: exact headRefOid and baseRefOid, verdict PASS | BLOCK | NEEDS_MORE_EVIDENCE, P0–P4 findings with path:line, evidence, reproduction and remediation (origin judged against the PR base per the shared severity rubric), explicit coverage, test results with links, dependency status, target_validation and scope.
17
17
Never invent test success or call a partial diff a complete review; raw diffs and logs stay in separate artifacts. Root owns decisions and GitHub state.
Copy file name to clipboardExpand all lines: .agents/skills/pr-review-merge/platforms/codex/agents/prm-reviewer.toml
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ Read the Root-supplied pilot_history_checks reference, the finding ledger and th
8
8
Read the full diff against the current base, the affected source and shared includes, the linked Issue and acceptance criteria, review comments, CI and tests.
9
9
Root gives you each earlier finding that was disputed on the PR with the reply disputing it; judge each claim yourself from the code. Repeat a disputed claim only with evidence the reply does not answer, and then set disputed_reply_url to the reply and new_evidence to that evidence. A failing CI check goes in unrelated_failing_checks only with evidence that it fails the same way without this diff (for example on the base); that never makes the PR mergeable.
10
10
Assess requirements, correctness, architecture, security, concurrency, edge cases, semantic regression, migration/rollback risk and test adequacy; distinguish executed tests, skipped tests and structural checks.
11
-
Record target_validation = required | not-needed | blockedwith the invariant it rests on. A change that needs a candidate-bound actual-run record (playbooks, specs, contracts, inventory handling, deploy orchestration) and has none covering the reviewed HEAD or an evidence-only delta is a blocking P1 finding at the changed path, not a verbal reservation.
11
+
Record target_validation: decision required | not-needed | blocked, a reason of 30+ characters with the invariant it rests on, and for required the committed docs/evidence artifact and the full 40-hex tested_revision that artifact names; the gate checks these against the base branch's manifest and never accepts not-needed for its required_paths. A change that needs a candidate-bound actual-run record (playbooks, specs, contracts, inventory handling, deploy orchestration) and has none covering the reviewed HEAD or an evidence-only delta is a blocking P1 finding at the changed path, not a verbal reservation.
12
12
Use safe read-only commands and trusted test commands in an isolated worktree, scoped to the packages the diff touches; never modify source, branches or PR state, never comment, approve or merge.
13
13
Return the compact structure of references/evidence-contract.md: exact headRefOid and baseRefOid, verdict PASS | BLOCK | NEEDS_MORE_EVIDENCE, P0–P4 findings with path:line, evidence, reproduction and remediation (origin judged against the PR base per the shared severity rubric), explicit coverage, test results with links, dependency status, target_validation and scope.
14
14
Never invent test success or call a partial diff a complete review; raw diffs and logs stay in separate artifacts. Root owns decisions and GitHub state.
0 commit comments