ci: only reviews run under the review contract count against the budget (ADR-1129) by cdeust · Pull Request #965 · cdeust/Cortex

cdeust · 2026-11-10T06:51:05Z

Summary

The review budget counts only reviews run under the review contract (ADR-1129, amends ADR-1119).

A pull request opened before a repository received the contract (the cap, then the Verdict: line the merge waits for) was reviewed by the earlier workflow, sometimes more than four times, with no verdict. Its budget was then already used: the review was skipped on every new head, no verdict could exist, and the resolver held the merge for ever. claude-rereview grants a fourth review, never a fifth, so it did not help.

Measured on 2026-11-10: ai-architect-mcp-spec #182, #184, #181 and ai-architect-mcp-codebase #484, #481, #478, #476. #182 had four earlier reviews; relabelled and reopened, its run 42769093930 still ended "3 of 3 reviews used".

ran_the_action now selects the PR Review step only in a run that has the Claude review budget job, which the earlier workflow did not have. The script still reads run metadata only.

Measured

Applied to the real runs of ai-architect-mcp-spec #182 with jq: the four earlier reviews went from counted to not counted. tests_py/scripts/test_ci_review_budget.py: 22 passed. ruff check, ruff format --check, scripts/check_craftsmanship.py: pass.

Generated by Claude Code

…et (ADR-1129) A pull request reviewed by the workflow that preceded the cap and the verdict line had its budget used by reviews that posted no verdict: the review was skipped on every new head and the merge was held for ever (ai-architect-mcp-spec #182, #184, #181; ai-architect-mcp-codebase #484, #481, #478, #476 on 2026-11-10). A run now counts only if it has the Claude review budget job. Co-Authored-By: Claude Fable 5.2 <[email protected]> Claude-Session: https://claude.ai/code/session_018kyZYsWnKSjpX4oNV19PCF Signed-off-by: Claude <[email protected]>

chatgpt-codex-connector · 2026-11-10T06:51:16Z

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-10T19:17:21.567279Z	`10be5f4`	New commits
🔒 Security Review	✅ Completed 2026-11-10T19:19:02.535241Z	`10be5f4`	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

claude · 2026-11-10T07:00:23Z

Claude finished @cdeust's task in 1m 41s —— View job

Review of PR #965 (review 1 of 3)

Read the diff, scripts/ci_review_budget.py and .github/workflows/claude-review.yml
Ran five reviews (code quality, security, test coverage, documentation, performance) and waited for their reports
Posted the review: ci: only reviews run under the review contract count against the budget (ADR-1129) #965 (comment)

No Blockers. I posted three Notes that need no action before merge.

Verdict: approve.

I could not run pytest, ruff or scripts/check_craftsmanship.py in this job because the commands needed approval. The PR body reports that they pass.

claude · 2026-11-10T07:01:57Z

Claude review 1 of 3: PR #965

I checked the diff with five reviews: code quality, security, tests, docs and performance. None of them found a Blocker.

jq filter (scripts/ci_review_budget.py:112-115): I checked it against the response of the "list jobs for a workflow run" endpoint.
- A run without the Claude review budget job gives [], so it is not counted.
- A run with that job and a successful PR Review step gives ["success"].
- steps[]? handles queued jobs that have no steps.
- The call has no --paginate, which is fine: the workflow has 2 jobs and a page holds 30.
Names: BUDGET_JOB matches .github/workflows/claude-review.yml:22, and a test pins it.
Security: job names come from the PR branch's copy of the workflow. Renaming the job there would stop a run counting, but renaming PR Review already did that on main, and so does deleting the skip condition. Fork PRs are excluded by the head.repo condition. So the change opens no new way around the cap.
Docs: the ADR mirror, the wiki page, the manifest entry and the CHANGELOG all match what the code does.
Not run here: I could not run pytest, ruff or the craftsmanship gate in this job (the commands need approval). The PR body reports that all of them pass.

Notes, no action before merge

test_the_budget_job_name_is_the_workflow_s is in the jq-gated JqFilterTests class (tests_py/scripts/test_ci_review_budget.py:163), so it is skipped when jq is missing even though it never calls jq.
No test covers a run that has the budget job but no PR Review step (expected []).
ADR-1129 Context says "sometimes more than four times", but its only example (fix(marketplace): bump withheld pins + CI pin-staleness gate #182) had exactly four.

Verdict: approve

cdeust · 2026-11-10T09:31:59Z

CI autopilot : #965 en retard de 4 commits sur main et en conflit (mergeable_state: dirty).

Le rebase sur origin/main (9c4eb98) bute sur un seul conflit, mécanique : wiki/manifest.json, où main a ajouté l'entrée ADR-1128 (#964) au même endroit que l'entrée ADR-1129 de cette branche. La résolution consiste à garder les deux entrées, dans l'ordre numérique. Je n'ai pas pu pousser ce rebase : le push avec --force-with-lease est refusé par les permissions de la session de l'autopilot, donc j'ai annulé le rebase local. La branche est inchangée (b528fc6).

Generated by Claude Code

Conflict: wiki/manifest.json, the ADR-1128 and ADR-1129 entries are both kept. The ADR-1130 entry is added: its decision file came with #975 without its manifest line, which made check_project_wiki.py fail on main. Co-Authored-By: Claude Fable 5.2 <[email protected]> Claude-Session: https://claude.ai/code/session_018kyZYsWnKSjpX4oNV19PCF Signed-off-by: Claude <[email protected]>

claude · 2026-11-10T19:23:04Z

Claude finished @cdeust's task in 1m 52s —— View job

Review 2 of 3 of PR #965

Check the Blockers from review 1: there were none
Find the commits pushed after b528fc6: only the merge 10be5f4 (main e2a9479 into the branch)
Run five reviews on that commit (code quality, security, test coverage, documentation, performance) and wait for their reports
Post the review: ci: only reviews run under the review contract count against the budget (ADR-1129) #965 (comment)

The merge kept the branch's change intact. In wiki/manifest.json it resolves the conflict by listing ADR-1128, 1129 and 1130 in order, and all the files those entries point to exist. None of the five reviews found a Blocker. I posted one Note.

I did not run pytest or ruff in this job. The review checked the code by reading it.

Verdict: approve

claude · 2026-11-10T19:25:08Z

Claude review 2 of 3: PR #965

There were no Blockers in review 1. Since b528fc6 the only new commit is the merge 10be5f4 (main e2a9479 into the branch), so I reviewed that and did not re-audit the unchanged code. I ran five reviews: code quality, security, tests, docs and performance. None of them found a Blocker.

The branch's change came through the merge intact. The diff against main is still the same 6 files, +106/−5. BUDGET_JOB (scripts/ci_review_budget.py:51) and ACTION_STEP (:46) still match .github/workflows/claude-review.yml:22 and :91.
wiki/manifest.json: the JSON is valid. The entries run in order ADR-1128, 1129, 1130, and all six files they point to exist. No conflict markers are left.
CHANGELOG.md: the ADR-1129 entry (lines 179-181) is under [Unreleased] / ### Changed, next to the other CI entries.
Security and performance: nothing changed. The budget job still runs main's script with read-only permissions, and there is still one jobs call per earlier run.

Notes, no action before merge

The merge also adds a manifest entry for ADR-1130. Main had that ADR's files but no manifest entry, so this PR fills the gap on main.

View job run
Verdict: approve

This was referenced Nov 10, 2026

ci: count only reviews run under the review contract; one review at a time cdeust/ai-architect-mcp-spec#188

Merged

ci: count only reviews run under the review contract; one review at a time cdeust/ai-architect-mcp-codebase#490

Merged

cdeust enabled auto-merge November 10, 2026 19:10

github-actions Bot merged commit cdcba71 into main Nov 10, 2026
18 checks passed

Navigation Menu

Repository navigation

ci: only reviews run under the review contract count against the budget (ADR-1129) - #965

ci: only reviews run under the review contract count against the budget (ADR-1129)#965
github-actions[bot] merged 2 commits into
mainfrom
ci/budget-counts-contract-reviews

cdeust commented Nov 10, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 10, 2026 •

edited

Loading

Uh oh!

claude Bot commented Nov 10, 2026 •

edited

Loading

Uh oh!

claude Bot commented Nov 10, 2026

Uh oh!

cdeust commented Nov 10, 2026

Uh oh!

claude Bot commented Nov 10, 2026 •

edited

Loading

Uh oh!

claude Bot commented Nov 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

cdeust commented Nov 10, 2026

Summary

Measured

Uh oh!

chatgpt-codex-connector Bot commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review of PR #965 (review 1 of 3)

Uh oh!

claude Bot commented Nov 10, 2026

Claude review 1 of 3: PR #965

Uh oh!

cdeust commented Nov 10, 2026

Uh oh!

claude Bot commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review 2 of 3 of PR #965

Uh oh!

claude Bot commented Nov 10, 2026

Claude review 2 of 3: PR #965

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chatgpt-codex-connector Bot commented Nov 10, 2026 •

edited

Loading

claude Bot commented Nov 10, 2026 •

edited

Loading

claude Bot commented Nov 10, 2026 •

edited

Loading