Skip to content

Layer-scoped host-receipt verification; version-aware supersession (GPT-6 + Opus verified) - #321

Merged
seathatflowsinourveins merged 1 commit into
mainfrom
claude/receipt-layer-scope-20260926
Sep 26, 2026
Merged

seathatflowsinourveins merged 1 commit into
mainfrom
claude/receipt-layer-scope-20260926

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

What was wrong. A GPT-6 cross-family review of #312 found that host-receipt verification wasn't scoped to layers:

  • scripts/component_matrix.py marked a catalog alternative host_verified whenever any independently reviewed use pass existed for its repository, whatever layer or role the receipt had exercised.
  • The Codex use receipt through its SDK surfaces, layer-scoped, with retained native outputs (GPT-6 reviewed) #312 Codex receipt exercises one read-only codex exec thread plus a resume, which is the native-clients role. It would have promoted the workers-layer alternative "Codex native workers", whose catalog text says the owned writing child is unproven.
  • The investigation also found that supersession was ignored: 26 superseded receipts still counted toward status.

What changes:

  • Scoped receipts: host_receipts.py record --layer-ref CATALOG/LAYER_ID (repeatable) writes the schema's existing optional layer_refs field. A scoped receipt counts only for winner cells and alternatives in the layers it names, and every platform_status caller now passes the row's layer.
  • Unscoped receipts: winners are unchanged. An alternative is verified by an unscoped receipt only when its repository appears in exactly one layer; otherwise the row stays receipts_recorded with layer_scope_needed: true.
  • record guard: record refuses a use receipt without --layer-ref when the component is in more than one layer. It exits with code 2 before running anything. Install and other stages are unaffected.
  • Supersession:
    • A receipt retires the nearest earlier generation recorded at its own version (pin-matching tool_versions[component_id]). It walks back past generations at other versions, so v1/agree → v2/agree → v1/needs_changes retires the first v1 receipt, and v2 keeps its own evidence.
    • A forked chain (two receipts superseding the same id) is rejected by validate and withholds acceptance from the fork on.
    • A superseded receipt is no longer acceptance evidence.
    • A superseded fail keeps blocking until a later, non-superseded native pass covers the same host, platform, pin, stage and layer.
    • Supersession is recomputed from whatever receipts the caller trusts, so a successor that the review gate filters out can't retire a base failure.
  • Review gate:
    • Every host receipt registered at the base must still exist at the head, keep its recorded claim, and keep the base's reviews as a prefix of its own.
    • The gate's base-trusted view now uses each base receipt's base snapshot. Before, it dropped any receipt whose bytes had changed, so appending a dissent to a blocking base failure removed that failure and let a same-PR promotion pass.
    • That bypass is pre-existing on main: it reproduces on the merge base without this PR's code, and this PR closes it.
    • Deleting or rewriting a dissented receipt can no longer revive an agreed predecessor. Appending a review still passes.
  • Fallback: host receipts under evidence/hosts/ no longer count through the generic evidence_refs fallback. They count only through the receipt checks.
  • Docs: docs/contributing-evidence.md and docs/next-host-stages.md document the field, the flag and the supersede rules, each with a "changed after v2026.09.26" marker.

Status changes (generated from the two component matrices)

Layer Alternative Before After Scope needed
foundation/ci-supply-chain gitleaks host_verified receipts_recorded yes
foundation/code-navigation codebase-memory-mcp host_verified receipts_recorded yes
foundation/native-clients MCPorter native bridge host_verified receipts_recorded yes
foundation/quality-evaluation Difftastic host_verified receipts_recorded yes
foundation/quality-evaluation MCP Inspector host_verified receipts_recorded yes
us-equities/agents-models-workers vllm host_verified receipts_recorded yes
us-equities/agents-models-workers Serena host_verified receipts_recorded yes
us-equities/agents-models-workers RTK host_verified receipts_recorded yes
us-equities/agents-models-workers foundation-codebase-memory-mcp host_verified receipts_recorded yes
us-equities/agents-models-workers Qdrant host_verified receipts_recorded yes
us-equities/evaluation-experiments promptfoo host_verified receipts_recorded yes
us-equities/identity-provenance foundation-socraticode host_verified receipts_recorded yes
us-equities/research-factors-ml foundation-qmd host_verified receipts_recorded yes
us-equities/research-factors-ml vllm host_verified receipts_recorded yes
us-equities/research-factors-ml foundation-markitdown host_verified receipts_recorded yes
Layer Winner Platform Before After

alternatives changed: 15; winner cells changed: 0

Of the 15 alternatives:

  • 8 are clear overclaims by the rows' own text:
    • MCPorter in native-clients;
    • SocratiCode in identity-provenance;
    • vllm, twice, where it is neither a worker nor a research method, and its passes are superseded;
    • Qdrant and RTK in agents-models-workers;
    • QMD in research-factors-ml, where there is no financial corpus;
    • gitleaks in ci-supply-chain, where the receipt was a local scan, not the CI requirement.
  • 7 plausibly did the same job but were recorded for another layer. One scoped re-record plus a review returns each one:
    • codebase-memory-mcp, twice;
    • Serena;
    • MCP Inspector;
    • Difftastic;
    • promptfoo;
    • MarkItDown.

No winner cell changes. An earlier revision of this PR demoted the vllm semantic-rag WSL cell, but that came only from a cross-version supersede link: -3 at 0.30.0 retired -2, which is independently agreed and bound to the 0.25.0 winner pin. Supersession is now version-aware, so -2's evidence counts again.

Rule choice. Here is how each candidate rule plays out on origin/main (183 alternatives, 19 host_verified):

Rule Alternatives still host_verified
R0: keep the current behaviour 19
R1: needs a scope when the repo is in more than one layer (chosen) 4
R2: needs a scope only when the repo wins another layer 6
R4: every alternative needs a scope 0

No rule based on catalog data alone separates the 8 overclaims from the 7 plausible rows. R2 would leave 32 multi-layer repositories that win no layer unscoped (codex-plugin-cc, temporal and toon, for example). The winner binding stays as it was: the 18 multi-layer winner cells each exercise the one job every selecting layer chose them for. For new receipts, the record guard closes the claude-code and codex multi-role case.

Review

  1. GPT-6 review of the original change: Codex gpt-6-astra, effort max, reviewed the first implementation and found 4 defects, all confirmed by reproduction:

    • HIGH: a successor in the same PR retired a trusted base failure through the review gate;
    • MEDIUM: superseded native fails were cleared by partial, synthetic or other-platform successors;
    • MEDIUM: host receipts could be accepted through the evidence_refs fallback;
    • LOW: a wrong claim about the v2026.09.26 schema.
  2. Repair: each defect became a failing regression test first (8 red), then was fixed.

  3. Status counts: unchanged between the two rounds.

  4. Verification round. A GPT-6 re-check and an independent Opus verifier (scratch export, reproductions and mutation tests) confirmed that all 4 round-1 findings are fixed. They found 3 more, all now fixed with failing-first regression tests:

    • medium: supersession ignored the component version, which caused the earlier vllm winner change;
    • medium/low: forked supersede chains passed validation;
    • low: a PR could delete a dissented base receipt to revive its agreed predecessor.
  5. Final round. A last GPT-6 check found 3 more defects. They are fixed with failing-first tests:

    • high, pre-existing on main: the gate's trust view dropped base receipts whose bytes changed, so an appended dissent removed a blocking failure;
    • medium: version matching looked only at the immediate predecessor;
    • low: wording. The us-equities vllm alternatives are demoted for missing layer scope, not supersession.

    Per the bounded-review rule, this was the last repair round. Anything later is recorded as a residual.

Verification

  • python -m unittest (full suite, after the commit, so the release-marker consistency test sees it): Ran 5845 tests, OK (skipped=613). An earlier run had 2 failures in untouched modules (gitleaks per-user lock contention and a 30 s subprocess timeout); both pass in isolation..
  • scripts/validate.py, host_receipts.py validate (all existing receipts still validate), component_matrix.py --check (0 flip-rule violations), new_host_grand_list.py --check, build_verdicts.py --check, evidence_manifest.py --check, build_ecosystem.py --check and validate_convergence.py --all-recorded: all pass.
  • verdict_review_gate.py: passed.
  • New-machine inputs changed after v2026.09.26; release_due.py reports release_due. 9a will cut one release covering the repair round.

Residuals

  • validate does not require layer_refs. Only record enforces the rule, so a hand-written unscoped receipt stays unscoped.
  • The matrix's raw receipt and dissented counts still include superseded receipts. They are display only.
  • Alternatives still ignore pins, as before.
  • Python 3.9 was checked by grammar parse only.
  • The 7 plausible rows need scoped re-records to return. That is follow-up work, not part of this PR.
  • record still accepts a cross-version --supersedes link. It retires nothing, and the docs say so.
  • validate does not compare platforms across a supersede link.
  • The gate fixes take effect only after this merges, because CI runs the base copy of the gate.

SOTA sources

🤖 Generated with Claude Code

@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 26, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@seathatflowsinourveins seathatflowsinourveins changed the title Layer-scoped host-receipt verification; superseded receipts stop counting (GPT-6 reviewed) Layer-scoped host-receipt verification; version-aware supersession (GPT-6 + Opus verified) Sep 26, 2026
@seathatflowsinourveins
seathatflowsinourveins force-pushed the claude/receipt-layer-scope-20260926 branch from 069dd22 to de92ca1 Compare September 26, 2026 05:01
A use receipt can name its layers (record --layer-ref CATALOG/LAYER_ID, the schema's layer_refs).
Scoped receipts count only for rows in those layers; an unscoped receipt verifies an alternative
only when that repository appears in exactly one layer, and record refuses an unscoped use receipt
for a multi-layer component. A successor retires its predecessor only at a pin-matching version;
forked supersede chains are rejected and withhold acceptance; a superseded fail keeps blocking until
a later native pass on the same host/platform/pin/stage/layer; host receipts no longer count through
the generic evidence_refs fallback; the review gate refuses deleting or rewriting a base-registered
host receipt. 15 of 19 host_verified alternatives return to receipts_recorded; no winner cell changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant