Require real behavior proof for external PRs#77622
Conversation
|
Codex review: needs changes before merge. Summary Reproducibility: yes. Source inspection of the PR head gives a high-confidence path: a Next step before merge Security Review findings
Review detailsBest possible solution: Land a maintainer-approved gate after tightening the classifier so evidence-type labels do not count without actual non-test live evidence, while preserving the trusted-base workflow design. Do we have a high-confidence way to reproduce the issue? Yes. Source inspection of the PR head gives a high-confidence path: a Is this the best way to solve the issue? No. The workflow shape is reasonable, but the classifier should reject mock/test/lint/typecheck/CI-only content before accepting evidence labels, or require actual live evidence payload beyond those labels. Full review comments:
Overall correctness: patch is incorrect Acceptance criteria:
What I checked:
Likely related people:
Remaining risk / open question:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 978bc53e80cc. Re-review progress:
|
81876f5 to
d094506
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
d094506 to
c4ed838
Compare
c4ed838 to
5eeb9c8
Compare
This adds the external-contributor proof gate we discussed. External PRs now have to fill a Real behavior proof section with the real setup, exact after-patch steps, evidence, observed result, and untested scope. The check deliberately treats tests, mocks, lint, typechecks, snapshots, and CI as supplemental, while accepting screenshots, recordings, terminal captures, console output, redacted runtime logs, linked artifacts, and copied live output.
Barnacle now labels missing or mock-only proof clearly and keeps that path passive until a maintainer uses the existing manual close trigger. The new required workflow check can be added to the main ruleset after this lands; the override remains the proof: override label instead of a ruleset exception.