Repository navigation
Top rule templates: carry the operator's 2026-09-28 top rule (X5a, X5c) - #458
Merged
seathatflowsinourveins merged 2 commits intoSep 28, 2026
Merged
Conversation
Lane A of the 2026-09-27 prompt audit's third round, decided by two independent lanes (GPT-6 through the packaged Codex runner, Claude through a blind-lane-reviewer) and, where they split, a blind adjudication in both presentation orders by both families. - examples/claude-native/CLAUDE.md:3 (X5a): both final-round lanes agreed. - adoption/templates/codex.AGENTS.template.md:3 (X5c): the final round split; all four adjudications chose the amendment that keeps the installed client as a source of truth. TOP_RULE_SHA256 and the word count (153) in tests/test_codex_worker_lane.py are recomputed with the test module's own template_segments(). - docs/harness-defaults.md: two anti-pattern rows (adjudicator input contract; void patterns proven only on synthetic cases). A third row, on print mode's wait for background agents, goes to #444 with the receipts of the run it describes. - docs/decisions/2026-09-28-top-rule-templates.md and evidence/artifacts/prompt-audit-20260927/lane-a/ (verify_lane_a.py recomputes both tallies and checks the lines and the pin). - One review round before the first push (GPT-6 through codex exec and a Claude evidence-reviewer, both changes-needed), one repair round and a re-check, recorded under lane-a/review/. Published scripts mask Claude subagent ids. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
deleted the
claude/template-user-level-sync-20260928
branch
September 28, 2026 12:14
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 28, 2026
… AGENTS.md; AN-13 receipt - CLAUDE.md lines 3-8 (X9): round 3 compared the two texts on the measure the second round named (native /mcp and /context refusal in print mode, with a promptfoo-graded, preregistered comparison) and the blind adjudication chose the GPT-6 lane's text in all four judgments. - AGENTS.md:3 (X5b): the operator's 2026-09-28 top-rule paragraph, byte for byte; AGENTS.md:37 (S1): build a PR description from the pull request template. Both converged in lane A (agree / agree). X5a and X5c land in #458. - AN-13: the headless /doctor prompt-audit run the 2026-09-28 community sweep left open. Facts of both runs in evidence/artifacts/prompt-audit-20260927/an13/ (the report text is not published); no edit made from it; each finding's disposition is in its README and in the decision record. - Decision record and x9-round3/ evidence (preregistration, isolation receipt, fixtures, judges, usage). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 28, 2026
…ith X5b, round-3 token receipts
- docs/harness-defaults.md: a dated row for running a headless skill
that starts background agents under print mode's default 600 s wait
(AN-13's first run: four Explore agents stopped by the client, an
interim note instead of the report); the rerun with
CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 returned the report
(code.claude.com/docs/en/env-vars).
- docs/harness-defaults.md:7 restates AGENTS.md's new top-rule heading
(X5b), disclosed in the decision record.
- controls/token-counts-r3-{f508ffb,base,after}.txt: the o200k counts
the record cites for CLAUDE.md (30 -> 96 at f508ffb; 96 -> 162 at
fb14ded) and AGENTS.md (2,374 -> 2,484 at fb14ded).
- The record cites validate.yml's heading rule at both revisions and
notes the Codex-lane host step done on this workstation after #458.
- x9-round3/README.md: the scan note names every kind of match (names,
not values), including one judgment's quote of the untested cases.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
5 tasks done
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 28, 2026
… AGENTS.md; AN-13 receipt - CLAUDE.md lines 3-8 (X9): round 3 compared the two texts on the measure the second round named (native /mcp and /context refusal in print mode, with a promptfoo-graded, preregistered comparison) and the blind adjudication chose the GPT-6 lane's text in all four judgments. - AGENTS.md:3 (X5b): the operator's 2026-09-28 top-rule paragraph, byte for byte; AGENTS.md:37 (S1): build a PR description from the pull request template. Both converged in lane A (agree / agree). X5a and X5c land in #458. - AN-13: the headless /doctor prompt-audit run the 2026-09-28 community sweep left open. Facts of both runs in evidence/artifacts/prompt-audit-20260927/an13/ (the report text is not published); no edit made from it; each finding's disposition is in its README and in the decision record. - Decision record and x9-round3/ evidence (preregistration, isolation receipt, fixtures, judges, usage). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 28, 2026
…ith X5b, round-3 token receipts
- docs/harness-defaults.md: a dated row for running a headless skill
that starts background agents under print mode's default 600 s wait
(AN-13's first run: four Explore agents stopped by the client, an
interim note instead of the report); the rerun with
CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 returned the report
(code.claude.com/docs/en/env-vars).
- docs/harness-defaults.md:7 restates AGENTS.md's new top-rule heading
(X5b), disclosed in the decision record.
- controls/token-counts-r3-{f508ffb,base,after}.txt: the o200k counts
the record cites for CLAUDE.md (30 -> 96 at f508ffb; 96 -> 162 at
fb14ded) and AGENTS.md (2,374 -> 2,484 at fb14ded).
- The record cites validate.yml's heading rule at both revisions and
notes the Codex-lane host step done on this workstation after #458.
- x9-round3/README.md: the scan note names every kind of match (names,
not values), including one judgment's quote of the untested cases.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 28, 2026
… AGENTS.md; AN-13 receipt - CLAUDE.md lines 3-8 (X9): round 3 compared the two texts on the measure the second round named (native /mcp and /context refusal in print mode, with a promptfoo-graded, preregistered comparison) and the blind adjudication chose the GPT-6 lane's text in all four judgments. - AGENTS.md:3 (X5b): the operator's 2026-09-28 top-rule paragraph, byte for byte; AGENTS.md:37 (S1): build a PR description from the pull request template. Both converged in lane A (agree / agree). X5a and X5c land in #458. - AN-13: the headless /doctor prompt-audit run the 2026-09-28 community sweep left open. Facts of both runs in evidence/artifacts/prompt-audit-20260927/an13/ (the report text is not published); no edit made from it; each finding's disposition is in its README and in the decision record. - Decision record and x9-round3/ evidence (preregistration, isolation receipt, fixtures, judges, usage). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 28, 2026
…ith X5b, round-3 token receipts
- docs/harness-defaults.md: a dated row for running a headless skill
that starts background agents under print mode's default 600 s wait
(AN-13's first run: four Explore agents stopped by the client, an
interim note instead of the report); the rerun with
CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 returned the report
(code.claude.com/docs/en/env-vars).
- docs/harness-defaults.md:7 restates AGENTS.md's new top-rule heading
(X5b), disclosed in the decision record.
- controls/token-counts-r3-{f508ffb,base,after}.txt: the o200k counts
the record cites for CLAUDE.md (30 -> 96 at f508ffb; 96 -> 162 at
fb14ded) and AGENTS.md (2,374 -> 2,484 at fb14ded).
- The record cites validate.yml's heading rule at both revisions and
notes the Codex-lane host step done on this workstation after #458.
- x9-round3/README.md: the scan note names every kind of match (names,
not values), including one judgment's quote of the untested cases.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 28, 2026
…CLAUDE.md, 4 of 4 blind judgments; dashboard manifest bound (#444) * AGENTS.md: scope role dispatch to new or ad-hoc stages; current-state holdout wording Two items of the 2026-09-27 prompt audit, resolved by cross-family convergence (docs/decisions/2026-09-27-prompt-audit-resolution.md, lane:foundation PR): - Role dispatch applies to each new or ad-hoc workflow agent() stage; the saved scripts vendored in examples/claude-native/workflows keep their reviewed routing, byte-identical to agent-lab, as that README states in its opening paragraph and the Workflow contract's Dispatch by role bullet (F1). - The inspected 2021 control segment: "has been inspected, so no experiment may present it as a fresh untouched holdout" (F4). o200k (gpt-tokenizer 3.4.0): AGENTS.md 2,374 -> 2,401 tokens (+27). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * X9 second round: blind two-family re-judgment splits again; CLAUDE.md unchanged After round 1 split 2 and 2 on X9 (CLAUDE.md:3-4), a second round added: - the installed client's own refusals of /context and /mcp through the Skill tool; - an executed comparison of the three texts, all headless claude -p runs through promptfoo 0.123.1: K1-K3 as 27 preregistered runs on Opus 5.5, and K4 as 9 runs on Sonnet 5; - 38 dated sources, plus 5 recorded searches that found nothing. Attempt 1 was void, because its judges could read this record and the coordinator's work directory. Attempt 2 was final. Its judges read a plain export of ba1700a and judged in both orders: blind-adjudicator agents and the packaged GPT-6 runner. A void audit, hashed before any return was read, found no voiding hit. Both GPT-6 judgments chose the GPT-6 lane's text, and both Claude judgments chose the Claude lane's text, so under the unanimity rule nothing is applied. All eight judgments across both rounds reject the current line. The addendum records both positions and the comparison that would decide it. Review: - One round, by GPT-6 and Claude reviewers, then one repair round: - disclose that K4 ran on Sonnet 5 while the packets said Opus 5.5; - make check_orders.py exit 1 on a difference, and add retally.py; - correct the counts; - add receipts (run windows, settings file names per run, the K4 skill-list sentences, the isolation receipt, a source supplement); - fix the wording. - A separate re-check found all 13 items fixed and two new defects: an overstated "keeps only" claim and an overstated account of K4's origin. Both are fixed. The recovered first check_orders.py is kept to show its flaw. - A pre-handoff review found that the published packets, prompts and K4 files quoted answer heads containing client configuration (an enabled-plugin entry, status-line wiring, plugin install records). Every published copy now withholds them. attempt2/sent-sha256.json keeps the hashes of the files as sent. This commit squashes the earlier ones, so no commit on the branch carries that text. docs/harness-defaults.md gains three dated anti-pattern rows: - a model scope taken from the preregistration instead of each run's recorded model; - a check that reported a mismatch and exited 0; - judge inputs published with answer text that quotes client configuration. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Prompt audit round 3: X9 applied to CLAUDE.md; lane A's S1 and X5b in AGENTS.md; AN-13 receipt - CLAUDE.md lines 3-8 (X9): round 3 compared the two texts on the measure the second round named (native /mcp and /context refusal in print mode, with a promptfoo-graded, preregistered comparison) and the blind adjudication chose the GPT-6 lane's text in all four judgments. - AGENTS.md:3 (X5b): the operator's 2026-09-28 top-rule paragraph, byte for byte; AGENTS.md:37 (S1): build a PR description from the pull request template. Both converged in lane A (agree / agree). X5a and X5c land in #458. - AN-13: the headless /doctor prompt-audit run the 2026-09-28 community sweep left open. Facts of both runs in evidence/artifacts/prompt-audit-20260927/an13/ (the report text is not published); no edit made from it; each finding's disposition is in its README and in the decision record. - Decision record and x9-round3/ evidence (preregistration, isolation receipt, fixtures, judges, usage). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Prompt audit: print-mode anti-pattern row (AN-13), line-7 co-change with X5b, round-3 token receipts - docs/harness-defaults.md: a dated row for running a headless skill that starts background agents under print mode's default 600 s wait (AN-13's first run: four Explore agents stopped by the client, an interim note instead of the report); the rerun with CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 returned the report (code.claude.com/docs/en/env-vars). - docs/harness-defaults.md:7 restates AGENTS.md's new top-rule heading (X5b), disclosed in the decision record. - controls/token-counts-r3-{f508ffb,base,after}.txt: the o200k counts the record cites for CLAUDE.md (30 -> 96 at f508ffb; 96 -> 162 at fb14ded) and AGENTS.md (2,374 -> 2,484 at fb14ded). - The record cites validate.yml's heading rule at both revisions and notes the Codex-lane host step done on this workstation after #458. - x9-round3/README.md: the scan note names every kind of match (names, not values), including one judgment's quote of the untested cases. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Grand dashboard: the evidence manifest gets its own read bound This pull request's registrations take manifests/evidence.json past the 2,000,000-byte bound progress.py applied to every source (1,987,653 bytes on main at fb14ded; 2,017,827 with this branch), so the snapshot refused it and eight dashboard tests failed. The snapshot reads only the manifest's receipt count: SOURCE_MAX_BYTES gives the manifest 8,000,000 bytes and every other source keeps MAX_BYTES. read(root, relative) keeps its signature (the tests patch it). The new test fails on the unchanged reader (controls/dashboard-bound-before.txt) and passes with 16 others (dashboard-bound-after.txt). Overturn: when the manifest nears the new bound, count receipts from a smaller source instead of raising it again. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Prompt audit round 3: review repairs, the audit's control, the Codex lane host receipt One review round on 0779e91a (GPT-6 codex exec and a Claude evidence-reviewer, both changes-needed); every finding was in the record or the evidence text: - the record: --max-turns 10 with one 11-turn result; every builder change after the freeze; the applied X9 text ties with the line it replaced on the preregistered metrics; the runs used ba1700a's files; X5b's wording and the tests that read AGENTS.md; the advisor calls; M4 is project-side; a review subsection; - x9-round3: audit_control_r3.py and judges/audit-control.json (clean copies reproduce the published verdicts; one planted access per judgment voids all four); the README lists every builder change, the judge-action flag and the tested configuration; - codex-worker-lane-host-20260928: the 12:14-12:15Z dry run, apply and proof, sanitized as #406's were, and a recheck with exit codes and times (dry run: block in place; proof 7 of 7); - review-444: both prompts, returns, usage and each finding's disposition. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Prompt audit: map the reviewed commits to the rebased ones The branch was rebased onto 9f8db58 (#400) with every patch unchanged (git range-diff shows each pair as =). review-444/README.md says 0779e91a, the reviewed head, was never pushed and was 7e5eddcd plus a registration commit, and gives the rebased SHAs of 9e036e7d and 7e5eddcd; the decision record points there. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Prompt audit: #446 carried the dashboard bound; map reviewed commits by subject After the rebase onto 3058b23, the dashboard commit keeps only its record subsection and the two receipts: #446 landed the same progress.py and test hunks (two-file patch-id 8f693abd4c601190). The review README now names the reviewed commits by subject, since each rebase changes their SHAs. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * AN-13: point M4 and M5 to their 2026-09-28 second-family lanes M4: both lanes reject the rewrite (no edit; the preload stays with the 2026-10-25 comparison). M5: the lanes amended with different texts and the blind adjudication split (unchanged). Both are recorded in docs/decisions/2026-09-28-an13-m4-m5.md, a separate lane:foundation PR. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * AN-13 M4 pointer: the trial's removal rule is open (#460) #460's record now says the M4 lanes did not test the skills trial's rule that removes a trial skill whose instructions conflict with CLAUDE.md or AGENTS.md, and leaves that question open. Say so where this record points to it. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Register the instruction files, the decision record, the anti-pattern rows and the prompt-audit evidence Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Scout <scout@local> Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
85a604aclane:foundation. Themanifests/evidence.jsonedit only registers this change's files, in the last commit.examples/claude-native/CLAUDE.md,adoption/templates/codex.AGENTS.template.md,tests/test_codex_worker_lane.py,docs/harness-defaults.md(two log rows),docs/decisions/2026-09-28-top-rule-templates.md,evidence/artifacts/prompt-audit-20260927/lane-a/(145 files),manifests/evidence.jsonWhat changes
examples/claude-native/CLAUDE.md:3(X5a):**Top rule: research convergence first; current upstream SOTA is the source of truth.** The installed client is also a source of truth; never self-write without a SOTA source. The ecosystem compounds: each choice adopts the current best converged practice and is replaced when the live landscape converges on a better-evidenced one.adoption/templates/codex.AGENTS.template.md:3(X5c): the same line without the bold markers.TOP_RULE_SHA256becomesce957fd8…and the pinned word count 153. Both are recomputed with the test module's owntemplate_segments().How each text was decided:
docs/harness-defaults.md: two dated anti-pattern rows:A third row, on print mode's wait for background agents, moves to Prompt audit rounds 2-3: AGENTS.md F1, F4, S1 and X5b; X9 applied to CLAUDE.md, 4 of 4 blind judgments; dashboard manifest bound #444 with the receipts of the run it describes.
Lane A's other items, S1 and X5b, change
AGENTS.md. They land with Prompt audit rounds 2-3: AGENTS.md F1, F4, S1 and X5b; X9 applied to CLAUDE.md, 4 of 4 blind judgments; dashboard manifest bound #444 (lane:shared).SOTA sources
lane-a/round1/user-level-top-rule.txtretains it.CLAUDE.mdfiles load. The portable template is installed as a user-level file.rust-v0.157.1codex-rs/core/src/agents_md.rs: how Codex loadsAGENTS.md. The installed client iscodex-cli 0.157.1.docs/harness-defaults.md#upstream-verification-and-compounding-learning. Line 58 names the installed client as a source of truth.docs/convergence-architecture.md;tools/sota-convergence/landscape-sweep/codex_call.sh(gpt-6-astra, effort max, read-only);.claude/agents/{evidence-reviewer,blind-lane-reviewer,blind-adjudicator}.md;docs/acceptance-evidence-policy.md.tools/token-report/token_manifest.py count_fileswith gpt-tokenizer 3.4.0 (o200k_base).Evidence-class table
source_review(retained model judgments)lane-a/final/{gpt6,claude}.jsonandtally-final.json, recomputed bylane-a/verify_lane_a.pysource_review(retained model judgments)lane-a/final/adjudication/{gpt6,claude}.{AB,BA}.json,audit-final.jsonandtally-adjudication-final.json, recomputed byverify_lane_a.pylane-a/controls/controls.json: exit 1, then exit 0controls.json: exit 1, then exit 0verify_lane_a.pychecks the tallies, both template lines and the pinmainbefore the change it reports 2 mismatches and exits 1 (lane-a/review/verify-negative-control.txt)lane-a/review/placeholder-check.json. Its failing run, with one planted line, isplaceholder-check-control.txtlane-a/review/exposure-scan-{before,after}.txt).git log -p origin/main..HEAD: 0 ids; the pre-repair commit had 6mainat4a610e18(lane-a/review/main-host-failures.txt)Local commands run
On the head of this PR (
6025b293):Decision record
docs/decisions/2026-09-28-top-rule-templates.mdnames the evidence, the alternatives considered and the comparison that would overturn the decision.Review
One review round ran before the first push, followed by one repair round and one re-check. The records are in
lane-a/review/, and the usage is in the decision record's Review section.GPT-6: one
codex exec -s read-only -m gpt-6-astra -c model_reasoning_effort=max -c web_search=live(codex-cli 0.157.1). Changes-needed, 5 findings.Claude: one
evidence-reviewer(Opus 5.5, effort max). Changes-needed, 8 findings.Repair: every finding was addressed;
lane-a/review/README.mdmaps each one to its repair. The material ones:verify_lane_a.py, which had no failing run.Re-check: the same Claude reviewer, resumed once, found every finding repaired and four open items. I resolved all four afterwards, and no reviewer has re-reviewed those resolutions:
The id fold is checked mechanically:
git log -pfinds no id in this PR's commits and 6 in the pre-repair commit.placeholder_check.pywas added after both reviews. Its failing control is recorded, but no reviewer has seen it.Residuals, recorded and not repaired:
docs/harness-defaults.md:7restates the repository top rule's old heading and changes with X5b in Prompt audit rounds 2-3: AGENTS.md F1, F4, S1 and X5b; X9 applied to CLAUDE.md, 4 of 4 blind judgments; dashboard manifest bound #444;Host step after merge
A Codex home that installed the lane block keeps the old line until
tools/adoption/apply_codex_lane.pyis run there again. Run a dry run first, then the--applycommand the dry run prints, with its--expect-*hashes (--applywithout them exits 2). The script refuses while Codex runs.Checklist
permissions: contents: read(or a narrower, explicitly justified addition). (None changed.)🤖 Generated with Claude Code