Skip to content

U11 design (for review): merit-neutral selection rules for the sweep refuter and verdict eligibility - #588

Merged
seathatflowsinourveins merged 4 commits into
mainfrom
foundation/u11-merit-neutral-design-20261001
Oct 1, 2026
Merged

seathatflowsinourveins merged 4 commits into
mainfrom
foundation/u11-merit-neutral-design-20261001

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator

Scope

  • What this PR changes: adds one design record for review, docs/decisions/2026-10-01-u11-merit-neutral-selection.md. It proposes rules that remove the incumbent's advantage from the sweep's fit refuter and from verdict eligibility. Nothing is implemented; no prompt, template, sealed verdict, active workflow or acceptance record changes.
  • Base commit: a2c22b26753e1fb6563b5d482fcba966a8a52310
  • Lane: lane:foundation
  • Owned paths touched: the one decision record.

This is a draft for the cross-family review coordinated by the Codex catalog lane's root and one Claude Opus review. The review question is at the end of the record. The implementation comes in a later pull request, with failing tests first.

SOTA sources

Evidence-class table

Claim Evidence class Command / receipt
The four rules exist as quoted source_review the files named above at a2c22b26
The proposed rules remove the advantage none: a design for review the review question in the record

Local commands run

$ python3 scripts/validate.py --scan-file docs/decisions/2026-10-01-u11-merit-neutral-selection.md   # passed

Decision record

docs/decisions/2026-10-01-u11-merit-neutral-selection.md (this pull request).

Host evidence

Not relevant.

Checklist

  • No workflow or action changes.
  • No secrets are printed, logged or committed.
  • No new paid hosting, subscription or billing surface was introduced.
  • Peer-owned untracked files and worktrees were preserved.

🤖 Generated with Claude Code

…y review

Four rules in the sweep and verdict tooling give the incumbent an advantage that repository quality does not: the fit
refuter's gap-versus-winners test with refute-when-uncertain, one list for discovery novelty and merit eligibility,
adopted-only verdict winners with no undetermined return, and evidence collected mostly on installed tools. The
record proposes a frozen eligible field per layer, a fit refuter v2 with a pending vote, a verdict contract v2 with
undetermined and one evidence bar for every candidate, and the comparison rules that decide. Nothing is implemented
and nothing live changes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Oct 1, 2026
Scout and others added 3 commits October 1, 2026 13:26
… and the Gate A owner's review

One field per layer that every repository sweep proposal enters, refuted ones as pending; a closed exclusion list
applied to every member; discovery admits against the frozen requirement; the fit refuter reads a blind projection and
excludes only when both families agree on a closed criterion; pending survives the converter and the ledger. Verdict
v2 seals adopted, retires challenger_preferred, gives each lane its own order seed, freezes aggregation across swapped
orders and panels, and enforces one evidence bar in the validators: a winner needs a measured comparison receipt that
covers the field. Comparisons freeze the arm-set reduction and an adoption-blind decision rule; ties never default to
the incumbent. The Gate A wiring arms are kept apart. Nothing is implemented.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ds of revision 2

From the Codex lane's read: no unconditional claim that the evidence bar makes a leak harmless; adoption withheld along
any judgment path that affects a deciding score; oracle and judgment evidence labelled apart; consistency is not
accuracy. From the Gate A owner's read: the validator recomputes the preregistered decision rule from the receipt and
the receipt cites a preregistration that predates its first trial; a provisional, labelled install for layers with
pending members or no comparison yet; requirement texts that name a candidate are reduced; three judgments from the
two routed families under different seeds; the residual rules and two corrections in Why. The trading lane owner's
answer on the pinned destination is quoted in E.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…cations after the delta-reads of revision 3

The Codex lane's delta-read: before a comparison, an incumbent install on a comparison layer is one isolated arm with
no operational or default precedence; a label alone does not satisfy decision 5. The Gate A owner's delta-read: the
receipt schema with per-arm results and the preregistration's hash and time in the files list; D's install order; two
judgments per family with agreeing majorities; one blind screen for every member's requirement-fit evidence and no
reference pair; the pin carve-out's overturn and the answer's source; the exposure measure; Why 7's six layers.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@seathatflowsinourveins
seathatflowsinourveins marked this pull request as ready for review October 1, 2026 19:23
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@seathatflowsinourveins
seathatflowsinourveins merged commit c45c9f1 into main Oct 1, 2026
25 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the foundation/u11-merit-neutral-design-20261001 branch October 1, 2026 19:24
seathatflowsinourveins added a commit that referenced this pull request Oct 17, 2026
…302; hold srt at 0.0.79

Claude Code 2.1.302 is the latest release. Its signed manifest
(https://downloads.claude.ai/claude-code-releases/2.1.302/manifest.json,
sha256 d7d4f193bfebb48733027b0dc43d6996d99f5335d152961a729730e55bbd5aa1)
verifies with gpg against https://downloads.claude.ai/keys/claude-code.asc
(fingerprint 31DD DE24 DDFA B679 F42D 7BD2 BAA9 29FF 1A7E CACE) in an empty
keyring. Its linux-x64 checksum, 60d7eb7f...29a5, matches the installed
versions/2.1.302 and the retained copy under claude-retained/2.1.302.

Function check from 2.1.300: the two `--help` texts are byte-identical,
and the exact claude_flags() list, run with no key from a scratch config
directory, gives the same init record (tools Glob, Grep, Read; no MCP
server; no plugin; permission mode dontAsk). The CHANGELOG entries for
2.1.301 and 2.1.302 (anthropics/claude-code@03b5892e and @c017428c)
change none of the flags, settings or the permission mode the worker uses.

The README's rollback pair moves to 2.1.300 (674deba0...5453, size
266972344, from its signed manifest).

srt stays at 0.0.79. 0.0.80 (anthropics/sandbox-runtime@626a5967) refuses
the sandboxed command new user namespaces by default (9a277e5, #588;
README.md:715 at the tag), and the worker's inner bwrap --unshare-user
then stops: boundary_self_check passes on 0.0.79 and fails on 0.0.80.
Whether to adopt the allowNestedUserNamespaces opt-out is the command
center's decision; worker.py records the finding next to the pin.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins added a commit that referenced this pull request Oct 17, 2026
…Claude Code (2.1.302) (#1127)

* Run the review worker on a retained, hash-checked copy of Claude Code 2.1.299

Native auto-update is on here (claude doctor, 2026-10-15T00:44:57Z: Auto-updates
enabled, channel latest). The native installer deletes old versions from
~/.local/share/claude/versions/, keeping only the launcher's version, any
version a running session uses and the two newest
(https://code.claude.com/docs/en/setup, home directories on network
storage). So the worker's pinned versions/2.1.299 path can vanish at the
next release.

- DEFAULT_CLAUDE_BIN is the command center's retained copy outside
  versions/, ~/.local/share/claude-retained/2.1.299/claude.
- DEFAULT_CLAUDE_BIN_SHA256 is 2d894a4b..., the linux-x64 checksum of the
  GPG-verified signed 2.1.299 manifest (manifest sha256 5052df4b...; good
  signature, fingerprint 31DD DE24 DDFA B679 F42D 7BD2 BAA9 29FF 1A7E CACE).
  The retained copy and versions/2.1.299 both hash to it.
- Preflight hashes the client binary at every tick and refuses one that
  differs, before any key is read. CLAUDE_BIN_SHA256 overrides the digest; a
  CLAUDE_BIN override is checked only when it is set. A malformed digest is
  a ConfigError.
- The vendor's other option, a custom launcher (which keeps every version),
  is not taken: it would change the binary every other session runs.

Tests: 96, OK (1 skipped, as on main). Against main's worker the settings
test fails and the new preflight test errors in all three subtests. The
real retained binary passes the check through file_sha256 (0.24 s).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Move the review worker to the retained, hash-checked Claude Code 2.1.300

On 2026-10-14 the owner directed (paraphrased, through the command center)
that the headless roles follow the latest Claude Code: pinned roles move
to each release through a retained, hash-checked copy after a function
check of their exact invocation, not a paired quality run.

- DEFAULT_CLAUDE_BIN is ~/.local/share/claude-retained/2.1.300/claude.
  DEFAULT_CLAUDE_BIN_SHA256 is 674deba0...5453, the linux-x64 checksum of
  the GPG-verified signed 2.1.300 manifest (manifest sha256 fb62dae2...).
  The retained copy and versions/2.1.300 both hash to it. The preflight
  check is unchanged.
- Function check, 2026-10-15, run against retained 2.1.299 and 2.1.300:
  - claude_flags(): every flag is present in both --help texts. The only
    change is --restricted's description, which drops REPL from the tools
    it removes; the worker passes --tools Read,Glob,Grep.
  - The exact flag list, run with no key from a scratch config dir, gives
    the same init record on both: tools Glob, Grep and Read, no MCP
    server, permission mode dontAsk.
  - The offline probe suite could not run on this host: srt 0.0.79 is not
    installed, since the worker is not enabled yet. That stays #1107's
    live-probe step before enabling.

Tests: 96 OK (1 skipped, as on main). With the previous worker.py the
settings test fails.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Fail closed on a CLAUDE_BIN override without its digest; record hash-read errors; recheck the pin at the run

Fix round 1 for #1127 (GPT full1 blocker 1, CC micro F1-F3, GPT P3-3 and P3-4):

- F1 / blocker 1: settings_from() raises ConfigError when CLAUDE_BIN
  names another path than the default and CLAUDE_BIN_SHA256 is unset or
  empty, so the command line exits 2 before a worker, a launcher or a
  GitHub call exists. The default path, however it is spelled, keeps the
  built-in digest. An explicit path/digest pair still works.
- F2 / P3-3: an OSError from the hash read (unreadable, or removed after
  the is_file check) is a "not ready" problem on the tick's own record
  path: a log line and a ticks.jsonl row, not a traceback.
- F3: run_keys() compares the run's re-hash with the pin before its
  first debit; a binary replaced after the tick's check starts nothing
  and debits nothing (no_stream: not counted, the tick stops).
- P3-4: the README records the rollback to the retained 2.1.299 copy
  (path and the 2.1.299 release manifest's linux-x64 checksum, with the
  checks before enabling) and its undo; the override note now says an
  override needs its digest. The test comment names 2.1.300.

Fail-first: the four new or changed tests failed at e6046ea (7
failures/errors, each for the defect: no ConfigError, None digest, a
worker built, a PermissionError traceback, a launch and two ledger rows
after the binary changed) and pass here; the module runs 99 tests OK.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Hash the client binary afresh before every review run

GPT read of #1127 fix1 at 795232e (P2), the same finding as the local review's P2 at worker.py:2054-2055:
binary_identity() cached the digest by (path, size, mtime_ns), so a replacement keeping size and
mtime (shutil.copystat copies nanosecond timestamps: python/cpython@cbc944f4 v3.13.16
Lib/shutil.py:348,385) passed the pin check with the old digest. run_keys() now calls
binary_identity(fresh=True), which hashes the file every run, and refuses with nothing debited when
the file cannot be read.

Regression test_a_replacement_keeping_size_and_mtime_is_caught_by_a_fresh_hash warms the cache with a
first review, replaces the binary with same-size bytes at the same mtime_ns, and expects no second
launch or debit. Fail-before: Tuples differ: (2, ['debit', 'settle', 'debit', 'settle']) !=
(1, ['debit', 'settle']). After: tests.test_claude_review_worker 100 tests OK (skipped=1).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Review worker: hash the binary before each key's launch; a receipt survives an unreadable binary

CC adjudication of #1127 at 0fb64d8 (the final round):
- (a) both families' P2 (worker.py:2056-2057): the run hashed the binary once, so a binary replaced while key 1
  was refused for credit was launched under key 2. Before each further key, run_keys hashes the file again; a
  mismatch or a read failure stops the run before the switch row, the debit and the launch.
  test_a_binary_replaced_before_the_next_key_starts_nothing_more, fail-before: Lists differ:
  ['anthropic-api-4', 'anthropic-api-3'] != ['anthropic-api-4'].
- (e) Astra P3s: the read-error refusal of fix 2 gets its own test, and the receipt's binary field reads the file
  through readable_binary_identity(), so a binary that stays unreadable no longer escapes while the refusal's
  receipt is written. test_a_binary_that_cannot_be_read_at_the_run_is_a_recorded_refusal, fail-before:
  PermissionError: [Errno 13] Permission denied.

After: tests.test_claude_review_worker 102 tests OK (skipped=1).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Move the review worker to the retained, hash-checked Claude Code 2.1.302; hold srt at 0.0.79

Claude Code 2.1.302 is the latest release. Its signed manifest
(https://downloads.claude.ai/claude-code-releases/2.1.302/manifest.json,
sha256 d7d4f193bfebb48733027b0dc43d6996d99f5335d152961a729730e55bbd5aa1)
verifies with gpg against https://downloads.claude.ai/keys/claude-code.asc
(fingerprint 31DD DE24 DDFA B679 F42D 7BD2 BAA9 29FF 1A7E CACE) in an empty
keyring. Its linux-x64 checksum, 60d7eb7f...29a5, matches the installed
versions/2.1.302 and the retained copy under claude-retained/2.1.302.

Function check from 2.1.300: the two `--help` texts are byte-identical,
and the exact claude_flags() list, run with no key from a scratch config
directory, gives the same init record (tools Glob, Grep, Read; no MCP
server; no plugin; permission mode dontAsk). The CHANGELOG entries for
2.1.301 and 2.1.302 (anthropics/claude-code@03b5892e and @c017428c)
change none of the flags, settings or the permission mode the worker uses.

The README's rollback pair moves to 2.1.300 (674deba0...5453, size
266972344, from its signed manifest).

srt stays at 0.0.79. 0.0.80 (anthropics/sandbox-runtime@626a5967) refuses
the sandboxed command new user namespaces by default (9a277e5, #588;
README.md:715 at the tag), and the worker's inner bwrap --unshare-user
then stops: boundary_self_check passes on 0.0.79 and fails on 0.0.80.
Whether to adopt the allowNestedUserNamespaces opt-out is the command
center's decision; worker.py records the finding next to the pin.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Review worker: srt 0.0.80 with nested user namespaces and a vsock check; absolute, pinned client paths

Fold of the CC verdict on #1127 at 37dfd70 (B1 to B8).

B1. srt moves to 0.0.80 (anthropics/sandbox-runtime@626a5967, tag v0.0.80).
Its seccomp helper refuses socket(AF_VSOCK) with EPERM
(vendor/seccomp-src/seccomp-unix-block.c:212-229; #691; absent at v0.0.79),
and a network namespace does not fence VM sockets on WSL2. Boundary.settings
sets allowNestedUserNamespaces (README.md:592, example :445-446;
src/sandbox/sandbox-config.ts:1179-1187), which lifts only the namespace
filter and the max_user_namespaces limit (apply-seccomp.c:887-888, :954-958),
so the inner bwrap keeps its user namespace and the socket filter stays.
Pinned from the registry: integrity sha512-WG4P8faU...MLpxnA== (npm view
dist), tarball sha256 7da66e1a5161bbcb57cf8a6f7ba7d85f1e6bb7897040b295885dc74c1a255f95,
the same from npm pack; npm audit signatures verifies it.
boundary_self_check gains a vsock line (socket creation only, never a
connect; only EPERM passes). Measured keyless on 2026-10-17: 0.0.79 fails
on vsock alone (the socket opens inside the chain); 0.0.80 without the
opt-out fails ("bwrap: No permissions to create a new namespace"); 0.0.80
with it passes, vsock refused EPERM. allowAllUnixSockets, which drops the
helper, joins SRT_FORBIDDEN. The README and the check's pass text say what
the boundary refuses.

B2. settings_from refuses a relative HOME and any path setting that is not
absolute after ~ expansion; Settings refuses a relative claude_bin, so the
hashed file and the bound file are one.

B3. README states that the hash check is by path, with its same-uid bound.

B4. Settings requires a 64-hex claude_sha256 (None or malformed raises
ConfigError), and the three hash gates no longer skip on a missing pin.

B5. A test for the per-key read-failure branch (again is None).

B6. The unreadable-binary test raises from the hash function instead of
chmod(0), which root ignores. An undebited refusal is recorded as an
uncounted attempt, so the next attempt gets its own number and receipt.
After a per-key stop the log names the key that ran and was refused.

B7. The override test states the lexical comparison and pins a .. spelling.

B8. The sha256-check test dates 2.1.302's adoption 2026-10-17.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Review worker: correct two comments (srt opt-out scope, Path comparison)

Comment-only fold of the CC delta verdict on #1127 at 738e24e.

B1. The srt pin comment also cites apply-seccomp.c:804-811 at v0.0.80:
under allowNestedUserNamespaces the helper's outer stub skips
PR_SET_NO_NEW_PRIVS as well as its namespace filter, and with no helper the
opt-out drops bubblewrap's --disable-userns
(linux-sandbox-utils.ts:3844-3852). The command still gets
PR_SET_NO_NEW_PRIVS (:943-945) and the socket filter. The v0.0.79 stub set
neither (apply-seccomp.c:764-769 at v0.0.79), so nothing widens against it.

B2. settings_from's comment and the settings test's comment say the
default-path test is a pathlib Path comparison: it folds repeated slashes,
`.` segments and a trailing slash, but not `..` or a leading `//`.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant