Skip to content

Gate A U9: independent frozen-check grader for the #381 token E2E (oracles, blind cross-family judges, controls) - #521

Merged
seathatflowsinourveins merged 64 commits into
mainfrom
claude/pra-u9-frozen-checks-2d-20260929
Sep 29, 2026
Merged

seathatflowsinourveins merged 64 commits into
mainfrom
claude/pra-u9-frozen-checks-2d-20260929

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator

Scope

  • What this PR changes, in one or two sentences: adds the independent frozen-check grader of the PR-H: E2E preregistration for the token-save-practice stack (frozen tasks, metrics, gate, procedure) #381 token E2E (grade.py with spec, bind, keys, capture, identity, grade, regrade, judge, controls, differential, export and check-html, over frozen_checks.py, evidence.py and judge.py, a Node bridge to the kernel, two schemas, a blind-judge template and Workflow, calibration controls and a unittest suite). Deterministic oracles grade what a file, a Git content or a transcript can decide; a blind judge from the other model family settles the semantic clauses. It changes no preregistration byte, no agent definition and no setting.
  • Base commit: 3c9438d6 (main after U8, U1, U13 and U1d)
  • Lane: lane:foundation (Gate A unit U9; owner native-agent-stack-2d)
  • Owned paths touched: tools/token-e2e/{frozen_checks,evidence,judge,grade}.py, tools/token-e2e/node_bridge.mjs, tools/token-e2e/frozen-check-judge.js, tools/token-e2e/{judge-template.md,judgment.schema.json,grading-block.schema.json}, tools/token-e2e/calibration/ (new), tests/test_token_e2e_grader.py (new), tests/fixtures/token_e2e_grader/ (new), tools/token-e2e/README.md (one grader section), docs/decisions/2026-09-29-frozen-check-grader.md (new), manifests/evidence.json (hot file, last commit)

SOTA sources

  • promptfoo 0.123.1 (npm view promptfoo, 2026-09-29; llm-rubric, g-eval) and inspect-ai 0.3.272 (PyPI, scorers): compared as the grading framework and rejected, with the deciding criteria (byte-identical results, keys surviving in results, a blind cross-family judge with isolation, quote checks and audit) in the decision record's table.
  • The TOON specification 4.1.1 and the installed TOON CLI 4.1.1 strict decode (the M7 round-trip check reads its exit status and text).
  • Upstream parts run unchanged: CPython ast, tokenize, json, hashlib, unicodedata; git; scripts/native_token_ci.py markdown_elements; the packaged Codex lane (tools/sota-convergence/codex_lane.py, transcript_audit.py) and the saved-Workflow pattern of adjudication-lane.js. No dependency is added.
  • The sealed preregistration and README of evidence/artifacts/token-adoption-e2e-20260926 (Amendment 3 seal d41152f4…; the R5 register, M9, the gates), read-only here. Claude Code 2.1.285 (read 2026-09-29).
  • Decision record with the comparison and the overturn conditions: docs/decisions/2026-09-29-frozen-check-grader.md.

Evidence-class table

Claim Evidence class Command / receipt
The grader module's own suite passes, with mutants for each guard local_integration (synthetic hosts, a scripted codex, recorded Workflow results, planted controls) python3 -B -m unittest tests.test_token_e2e_grader → Ran 624 tests in 278 s, OK (0 skipped)
Sibling suites still pass on the rebased head local_integration python3 -B -m unittest tests.test_token_e2e_preregistration tests.test_token_e2e_receipt_checks tests.test_skill_usage tests.test_token_measurement tests.test_freeze_snapshot → Ran 394 tests in 256 s, OK
Planted controls: A 79/79, B 20/20, C 17/17, D 21/21, E1 2/2; the T0 differential against the retained checks of two earlier runs agrees 4/4 and 4/4 local_integration (private host inputs reduced to counts) grade.py controls --spec <fixture spec> --repo <checkout>; grade.py differential --inputs <retained private inputs> (builder handoff, at the pre-rebase head)
spec refuses a preregistration without a grading block, as designed; with the sealed bytes (d41152f4…) plus a grading block it exits 0 twice with cmp equal local_integration grade.py spec at c0966da2 → E_GRADING_BLOCK field=missing, exit 2 (expected before Amendment 4); the two-run cmp on a scratch repo (independent re-verify)
Independent review, repair and re-verify source_review one Opus source review (2 high, 6 medium, 7 low, 12 gaps); one repair round fixed both highs and five mediums; the independent re-verify passed 13 of 14 findings and failed the T33 one (Residual 2)
The real judge routes (gpt-6-astra through the Codex lane, Opus through frozen-check-judge.js) return strict-schema judgments and pass their calibration controls NOT run (no model calls in this build; native and pooled GPT-6 capacity exhausted) `grade.py judge rehearse --route codex
GPT-6 cross-family review of the PR NOT run (capacity exhausted); recorded as a missing round -

Local commands run

$ python3 -B -m unittest tests.test_token_e2e_grader        (rebased head `5214f715`)
Ran 624 tests in 277.721s

OK
$ python3 -B -m unittest tests.test_token_e2e_preregistration tests.test_token_e2e_receipt_checks tests.test_skill_usage tests.test_token_measurement tests.test_freeze_snapshot
Ran 394 tests in 255.958s

OK
$ python3 scripts/validate.py
{"status": "passed", ...}
$ git diff --check 3c9438d6..HEAD
(no output, exit 0)

Builder record before the rebase (pre-rebase head fb2fa16c): Ran 624 tests in 342 s / OK, 0 skipped; the baseline of the reviewed head (705df138) ran 571 tests with 1 failure and 2 errors, all toon --encode/git timeouts at a host load average of 148 to 180 (environment, not code), and passed at lower load.

Residuals (bounded review loop: one review, one repair round; each is in the README limits or the decision record):

  1. The real judge routes were not run (see the table). Strict-schema acceptance by Codex, transcript-audit behaviour and calibration on real output stay unobserved; a real spec and keys run waits for Amendment 4.
  2. The T33 positive-control oracle (the re-verify's one failed finding): a correct answer that negates a level word in plain words other than its filler list, such as "No event has level ERROR.", "None of the five events is at ERROR level." or "ERROR: none.", still grades FAIL (levels) on both bounds of G-Q clause 1. A follow-up unit (U9b) replaces the fixed filler walk by a clause-scoped negation rule with failing-first tests, and must land before Amendment 4 is sealed.
  3. T32's facts and the post-decode web-table sum have no D-extract fallback, although the design (R3, R21) sends every unparsed key fact there, so wording the grammar misses (for example sum(latency_ms) = 124) grades unknown and costs the G-Q lower bound. U9b either adds the extraction controls per template or Amendment 4 records it as unknown by design. The builder's claim that D-extract may decide these was withdrawn.
  4. How real ai-memory results name the frozen records for T7 and T21 to T25 is unmeasured (MI-8); whole-path or digest matching could miss a result that shows only a title, which fails the task.
  5. Sibling shapes still read through fixtures: U2, U4, U10 and the U13 role rollouts. After U2 and U4 merge, NODE_FIXTURE_AT_BASE and the inside_git_work_tree allowance are removed in a consistency pass. U11's HTML is not built, so check-html on it is unobserved.
  6. Known low items: code_spans and _or_links can go quadratic on very long lines; toon_decode catches only ValueError; a nested bash -c wrapper can hide an encode from the M7 call classifier and cat <<EOF | toon -d is not counted as a child-side round trip; a web-table answer whose payload decoded and whose sum is unparsed can request a dead extraction packet (one wasted judge call).
  7. Behaviour change for operators: every grade.py parser sets allow_abbrev=False, so a flag must be written in full (an abbreviation is E_ARGS).
  8. The builder's freeze inputs named Claude Code 2.1.284; the host runs 2.1.285. The frozen version is set at the freeze (Amendment 4 input), not by this PR.
  9. Process: the first independent command re-run agent finished without calling StructuredOutput and counted as failed; the repair stage re-ran the acceptance list and the automatic re-verify followed. At one point two model loops of the same builder wrote in the worktree (a coordinator message reached a running builder); the builder resolved it through files and later stages verified the tree. Five early commit subjects were reworded in the builder's own unpublished history; five subjects on the branch exceed 72 characters (history is not rewritten; the PR is squash-merged).

Decision record

docs/decisions/2026-09-29-frozen-check-grader.md (framework comparison with the rejected alternatives promptfoo llm-rubric/g-eval and Inspect model_graded_*, the choices inside the grader and the overturn conditions).

Host evidence

Not applicable (no files under evidence/hosts/).

Checklist

  • New/changed GitHub Actions are pinned to a full commit SHA with a version comment (no workflow changes in this PR).
  • New/changed workflows declare top-level permissions: contents: read (no workflow changes).
  • No secrets are printed, logged or committed; fixtures use /home/example paths and no UUID-shaped values, private outputs are create-only mode 0600 outside every work tree, and the export canary refuses host paths.
  • No new paid hosting, subscription or billing surface was introduced.
  • Peer-owned untracked files and worktrees were preserved.

🤖 Generated with Claude Code

Scout and others added 30 commits September 29, 2026 18:58
Stage 1a of the U9 frozen-check grader (#381 token E2E): the tests come first, so each
guard and rule has a recorded failing value before tools/token-e2e/frozen_checks.py and
grade.py exist (1b).

Sources: the U9 build contract, sections a3, a4, b2 (R3-R5, R10, R12, R19, R20) and b5 (the
stage-1 list P0, F1-F7, F9, F10, F18, F20, F21, F25a, F26, F32, F33); the binding corrections
(item 2: the T0 re-run is the arm's own command in a temporary copy of the tree, because
`python3 -I -S` cannot import the tests package; item 3: the T14 query time comes from the
first tool_use naming the whole word agentsview, because the parser's closed program set
returns null for it); TOON spec v4.1.1 section 2 (equality); docs/acceptance-evidence-policy.md
"Discriminating controls". Each negative asserts its exact refusal code or rule id, never an
exit status alone: the missing script also exits 2, so the expected E_* line is what fails.

Also here, ahead of the b1 table, because 1b needs them: the stage-1 disarmed-guard mutants
(the F19 subset M1, M2, M3, M6, M10, M18, M19, M21, M22, M23, M24, each asserting the exact
set of tests it flips), the `keys` and `capture` command tests, and the schema and readings
table tests. Fixtures are generated at test time outside every work tree; no UUID-shaped
value or host path is committed.

Run at this parent (only U8 has merged; no grader module exists):
  TMPDIR=<scratch> PYTHONDONTWRITEBYTECODE=1 python3 -B -m unittest -v tests.test_token_e2e_grader
  Python 3.13.15, toon 4.1.1
  Ran 152 tests: FAILED (failures=13, errors=139)
Two tests pass at the parent by design: P0 test_preflight (a precondition; it records that
loadShellParser, shellParserStatus, commandInvocations, callLedger, m14State,
validateIdentityTable and skill_usage.inside_git_work_tree are absent, so fixtures of the
stated shape stand in) and P0 test_isolated_mode_cannot_import_the_tests_package (the control
that motivates correction 2). Every other test fails, with these values (one line per test;
"ModuleNotFoundError frozen_checks" is: ModuleNotFoundError: No module named 'frozen_checks'):

F10_Binding.test_a_path_alone_is_insufficient  ERROR ModuleNotFoundError frozen_checks
F10_Binding.test_inventory_and_line_count  ERROR ModuleNotFoundError frozen_checks
F10_Binding.test_no_sentinel_record  ERROR ModuleNotFoundError frozen_checks
F10_Binding.test_own_sentinel_passes  ERROR ModuleNotFoundError frozen_checks
F10_Binding.test_sibling_value_in_the_child_message  ERROR ModuleNotFoundError frozen_checks
F10_Binding.test_sibling_value_in_the_parent_message  ERROR ModuleNotFoundError frozen_checks
F18_LiteralAnswers.test_file_urls_and_templates_are_literals  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M10_spec_regeneration_off  ERROR ModuleNotFoundError grade
F19_Mutants.test_M18_create_only_off  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M19_work_tree_refusal_off  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M1_order_insensitive_headings  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M21_file_url_expansion  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M22_clause_check_off  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M23_key_drift_on_bytes  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M24_environment_dependent_mismatch_as_fail  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M2_one_equals_true  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M3_exclusions_off  ERROR ModuleNotFoundError frozen_checks
F19_Mutants.test_M6_no_strict  ERROR ModuleNotFoundError frozen_checks
F1_Inventory.test_amendment_one_fixture_counts  ERROR ModuleNotFoundError frozen_checks
F1_Inventory.test_sealed_preregistration_counts  ERROR ModuleNotFoundError frozen_checks
F20_GradingBlock.test_a_bogus_reading_is_refused_by_field  ERROR ModuleNotFoundError frozen_checks
F20_GradingBlock.test_a_changed_check_text_is_refused_through_the_cli  ERROR ModuleNotFoundError frozen_checks
F20_GradingBlock.test_a_missing_grading_block  FAIL refusal line: observed grade.py "can't open file" (exit 2), expected 'E_GRADING_BLOCK field=missing'
F20_GradingBlock.test_a_preregistration_that_differs_from_the_newest_seal  ERROR ModuleNotFoundError frozen_checks
F20_GradingBlock.test_a_stale_registry_hash_is_refused  ERROR ModuleNotFoundError frozen_checks
F20_GradingBlock.test_an_unknown_block_key_is_refused_by_field  ERROR ModuleNotFoundError frozen_checks
F20_GradingBlock.test_missing_amendment_four_heading  ERROR ModuleNotFoundError frozen_checks
F20_GradingBlock.test_no_expected_hash_override_flag  FAIL spec --help exit 2, expected 0
F20_GradingBlock.test_the_chosen_reading_is_used_and_the_alternatives_swap  ERROR ModuleNotFoundError frozen_checks
F20_GradingBlock.test_the_override_flag_is_rejected_with_a_refusal_code  ERROR ModuleNotFoundError frozen_checks
F20_GradingBlock.test_the_sealed_amendment_three_commit_has_no_block  FAIL refusal line: observed grade.py "can't open file" (exit 2), expected 'E_GRADING_BLOCK field=missing'
F20_GradingBlock.test_the_spec_writes_canonical_private_bytes  ERROR ModuleNotFoundError frozen_checks
F20b_SchemaAndReadings.test_a_fixture_block_validates_and_a_stale_field_names_its_path  ERROR ModuleNotFoundError frozen_checks
F20b_SchemaAndReadings.test_decided_first_then_the_ordered_alternatives  ERROR ModuleNotFoundError frozen_checks
F20b_SchemaAndReadings.test_the_committed_schema_equals_the_derived_one  ERROR ModuleNotFoundError frozen_checks
F20b_SchemaAndReadings.test_the_validator_refuses_a_keyword_it_cannot_check  ERROR ModuleNotFoundError frozen_checks
F21_SpecRegeneration.test_a_hand_edited_spec_is_refused  ERROR ModuleNotFoundError frozen_checks
F21_SpecRegeneration.test_the_unedited_spec_passes_the_regeneration_check  ERROR ModuleNotFoundError frozen_checks
F25a_MemoryKeys.test_an_empty_topic_blocks_launch  ERROR ModuleNotFoundError frozen_checks
F25a_MemoryKeys.test_only_records_before_since_with_the_anchors_are_frozen  ERROR ModuleNotFoundError frozen_checks
F25a_MemoryKeys.test_since_is_the_earliest_window_start  ERROR ModuleNotFoundError frozen_checks
F25a_MemoryKeys.test_the_keys_command_exits_2_with_the_task_and_writes_nothing  ERROR ModuleNotFoundError frozen_checks
F25a_MemoryKeys.test_topics_are_checked_in_sorted_order  ERROR ModuleNotFoundError frozen_checks
F26_Bind.test_a_missing_base_is_refused  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F26_Bind.test_a_missing_root_value_is_refused  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F26_Bind.test_a_sentinel_byte_change_is_refused  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F26_Bind.test_a_symlink_output_counts_as_existing  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F26_Bind.test_an_existing_output_is_never_overwritten  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F26_Bind.test_an_input_hash_mismatch_is_refused  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F26_Bind.test_an_output_inside_a_work_tree_is_refused_before_anything_is_created  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F26_Bind.test_an_unknown_key_is_refused  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F26_Bind.test_codex_bindings_are_merged_and_checked  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F26_Bind.test_run_record_args_that_differ_from_the_bind_mark_the_task  ERROR ModuleNotFoundError frozen_checks
F26_Bind.test_the_record_is_private_and_holds_the_token_only_as_a_hash  FAIL bind fixture: grade.py "can't open file" (exit 2), no bindings written
F27a_KeysCommand.test_an_existing_output_is_refused  ERROR ModuleNotFoundError frozen_checks
F27a_KeysCommand.test_keys_are_private_deterministic_and_record_missing_inputs  ERROR ModuleNotFoundError frozen_checks
F27b_CaptureCommand.test_a_failed_download_is_recorded_not_hidden  ERROR ModuleNotFoundError frozen_checks
F27b_CaptureCommand.test_a_window_capture_needs_a_family  ERROR ModuleNotFoundError frozen_checks
F27b_CaptureCommand.test_post_arm_discards_a_changed_tree_and_lists_new_processes  ERROR ModuleNotFoundError frozen_checks
F27b_CaptureCommand.test_post_arm_needs_the_pre_arm_capture  ERROR ModuleNotFoundError frozen_checks
F27b_CaptureCommand.test_post_window_runs_the_acceptance_commands_in_an_exported_tree  ERROR ModuleNotFoundError frozen_checks
F27b_CaptureCommand.test_pre_arm_captures_six_identities_twice_and_inventories_the_exec_checkout  ERROR ModuleNotFoundError frozen_checks
F27b_CaptureCommand.test_the_run_token_must_match_the_bind  ERROR ModuleNotFoundError frozen_checks
F27b_CaptureCommand.test_window_pages_are_captured_with_curl_fail_silent_show_error_location  ERROR ModuleNotFoundError frozen_checks
F2_ExactTextRegistry.test_digit_change_is_unmapped  ERROR ModuleNotFoundError frozen_checks
F2_ExactTextRegistry.test_dropped_task_must_be_listed  ERROR ModuleNotFoundError frozen_checks
F2_ExactTextRegistry.test_registry_has_52_exact_keys_for_40_templates  ERROR ModuleNotFoundError frozen_checks
F2_ExactTextRegistry.test_task_declared_by_no_template_is_refused  ERROR ModuleNotFoundError frozen_checks
F31a_T14QueryTime.test_a_grep_for_the_word_is_not_a_query  ERROR ModuleNotFoundError frozen_checks
F31a_T14QueryTime.test_heredoc_query  ERROR ModuleNotFoundError frozen_checks
F31a_T14QueryTime.test_lookalike_words_do_not_count  ERROR ModuleNotFoundError frozen_checks
F31a_T14QueryTime.test_no_query  ERROR ModuleNotFoundError frozen_checks
F31a_T14QueryTime.test_query_inside_ctx_execute_code  ERROR ModuleNotFoundError frozen_checks
F32_TokenLines.test_the_decided_reading_and_its_alternative  ERROR ModuleNotFoundError frozen_checks
F32_TokenLines.test_the_real_handbook_counts  ERROR ModuleNotFoundError frozen_checks
F32_TokenLines.test_three_counts  ERROR ModuleNotFoundError frozen_checks
F33_PageFacts.test_a_changed_retention_fact_is_drift  ERROR ModuleNotFoundError frozen_checks
F33_PageFacts.test_a_missing_header_token_is_key_missing  ERROR ModuleNotFoundError frozen_checks
F33_PageFacts.test_capture_records_status_bytes_and_hash_from_curl  ERROR ModuleNotFoundError frozen_checks
F33_PageFacts.test_different_bytes_with_the_same_facts_do_not_drift  ERROR ModuleNotFoundError frozen_checks
F33_PageFacts.test_extractors_find_the_frozen_facts  ERROR ModuleNotFoundError frozen_checks
F3_Clauses.test_every_clause_and_literal_is_verbatim  ERROR ModuleNotFoundError frozen_checks
F3_Clauses.test_mutated_clause_is_refused  ERROR ModuleNotFoundError frozen_checks
F3_Clauses.test_mutated_recipe_literal_is_refused  ERROR ModuleNotFoundError frozen_checks
F3_Clauses.test_web_table_first_clauses_are_class_d  ERROR ModuleNotFoundError frozen_checks
F4_Keys.test_e1_qmd_wrong_document_fails_t4  ERROR ModuleNotFoundError frozen_checks
F4_Keys.test_e1_repomix_answer_fails_t8_at_the_e1_keys  ERROR ModuleNotFoundError frozen_checks
F4_Keys.test_fixture_numbers  ERROR ModuleNotFoundError frozen_checks
F4_Keys.test_source_keys_at_the_pinned_revision  ERROR ModuleNotFoundError frozen_checks
F4_Keys.test_t1_heading_key_and_order_check  ERROR ModuleNotFoundError frozen_checks
F4_Keys.test_t3_definition_readings  ERROR ModuleNotFoundError frozen_checks
F4_Keys.test_t6_key_is_the_trust_segment  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_attribute_call_site_is_excluded  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_duplicate_site  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_exact_sites_pass  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_import_comment_and_string_sites_are_excluded  ERROR subtest: ModuleNotFoundError frozen_checks
F5_SymbolSites.test_import_comment_and_string_sites_are_excluded  ERROR subtest: ModuleNotFoundError frozen_checks
F5_SymbolSites.test_import_comment_and_string_sites_are_excluded  ERROR subtest: ModuleNotFoundError frozen_checks
F5_SymbolSites.test_missing_site  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_synthetic_key  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_t11_enclosing_function_readings  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_t5_labelled_imports_and_mentions_are_not_invocations  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_t5_precision_only_alternative  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_unrecognised_sum_is_unparsed_not_failed  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_wrong_events  ERROR ModuleNotFoundError frozen_checks
F5_SymbolSites.test_wrong_value_sum  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_a_found_but_undecodable_candidate_is_a_failure  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_an_evidence_string_is_a_candidate  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_decode_argv_never_carries_no_strict  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_every_candidate_must_equal_the_key  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_json_root_array_passes  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_missing_record  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_no_candidate_is_unknown_unparsed  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_seventeen_point_zero_equals_seventeen  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_single_key_wrapper_is_a_shape_failure_under_the_decided_reading  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_string_for_a_number_is_a_type_failure  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_toon_root_array_passes  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_true_for_one_is_a_type_failure  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_unfenced_payload_is_a_candidate  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_unordered_keys  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_wrapper_with_the_sum_needs_the_extras_reading  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_wrong_count_header_is_a_toon_strict_decode_failure  ERROR ModuleNotFoundError frozen_checks
F6_StrictDecode.test_wrong_latency_sum  ERROR ModuleNotFoundError frozen_checks
F7_T0.test_a_missing_capture_fails  ERROR ModuleNotFoundError frozen_checks
F7_T0.test_a_tree_changed_between_captures_discards_the_post_capture_and_never_reruns  ERROR ModuleNotFoundError frozen_checks
F7_T0.test_all_six_captures_and_the_answer_agree  ERROR ModuleNotFoundError frozen_checks
F7_T0.test_an_older_subject_first  ERROR ModuleNotFoundError frozen_checks
F7_T0.test_an_unverified_sandbox_makes_the_unittest_fields_environment_dependent  ERROR ModuleNotFoundError frozen_checks
F7_T0.test_environment_dependent_skip_count_is_unknown  ERROR ModuleNotFoundError frozen_checks
F7_T0.test_pre_and_post_disagreement_is_a_conflict  ERROR ModuleNotFoundError frozen_checks
F7_T0.test_test_count_mismatch_when_not_environment_dependent  ERROR ModuleNotFoundError frozen_checks
F7_T0.test_words_for_numbers_are_unparsed_until_d_extract_quotes_them  ERROR ModuleNotFoundError frozen_checks
F7b_T0Rerun.test_the_command_imports_the_tests_package_in_the_copy  ERROR ModuleNotFoundError frozen_checks
F9_Builder.test_bytes_differ_and_nothing_executes  ERROR ModuleNotFoundError frozen_checks
F9_Builder.test_empty_diff_at_the_prepared_path  ERROR ModuleNotFoundError frozen_checks
F9_Builder.test_extra_ignored_file  ERROR ModuleNotFoundError frozen_checks
F9_Builder.test_extra_untracked_file  ERROR ModuleNotFoundError frozen_checks
F9_Builder.test_mutated_fixture_hash_makes_the_key_unknown  ERROR ModuleNotFoundError frozen_checks
F9_Builder.test_observed_tree_other_than_the_prepared_path  ERROR ModuleNotFoundError frozen_checks
F9_Builder.test_only_before_py_changed_and_equal_to_after_passes  ERROR ModuleNotFoundError frozen_checks
F9_Builder.test_starting_revision_other_than_the_frozen_execution_revision  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t16_payload_and_citation  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t27_acceptance_summary  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t28_review_diff_counts  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t29_scout_inventory  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t2_recovery_digest_and_bytes  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t30_compile_exit  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t32_blind_verdict_id_and_latency  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t33_positive_control_events  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t34_main_output_count  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t35_agent_path_first_and_last  ERROR ModuleNotFoundError frozen_checks
G_TemplateSweep.test_t38_all_ten_structure_checks_are_read_from_the_answer  ERROR ModuleNotFoundError frozen_checks
P0_SiblingPreflight.test_isolated_mode_cannot_import_the_tests_package  ok (precondition)
P0_SiblingPreflight.test_preflight  ok (precondition)

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
First implementation step of stage 1b, in tools/token-e2e/frozen_checks.py and
grading-block.schema.json.

What: the 52-key exact-text registry mapping to 40 templates (RV-23), each template's
class set and its verbatim D clauses and recipe literals (E_CLAUSE), the a3 inventory
computed from the preregistration bytes and stamped with their sha256 (E_CHECK_UNMAPPED,
E_TEMPLATE_TASKS), the one readings table that feeds both the schema's enums and the
spec's alternatives (R2-01 gains wrapper_with_extras for the recheck's R4 finding), a
stdlib validator for the schema keyword subset that refuses any other keyword, the pinned
sources (git cat-file batch reads, tracked-file directory reads), the linear text scanners
of grammar g1, and create-only mode 0600 private writes refused inside a work tree.

Sources: repair-u9.design.md a3, a4, b2 R3, R5, R20, R22; JSON Schema 2020-12 validation
vocabulary (the keyword subset); the registry constants were generated from the sealed
bytes at c0966da (d41152f4), not typed.

Tests now passing (dev runner over the 857dc393 test file): F1_Inventory (2),
F2_ExactTextRegistry (4), F3_Clauses (4), F20b_SchemaAndReadings (4). Not yet built:
keys, oracles, captures, grade.py.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Both are defects in test data, not in an assertion's intent.

1. F5_SymbolSites built its synthetic repository once per subtest and per test in
   the same directory; the second `git commit` found nothing to commit and the test
   failed with "git commit failed". The key is now built once per test instance.
2. The synthetic enclosing function was named `caller`, which is also the word in
   the prose answer "Direct caller: ...", so the sites-only answer named the
   function by accident and the decided reading could not fail it. The function is
   now `use_registry`, a name no sentence uses.

Verified: F5_SymbolSites 12 tests OK with the implementation of the next commit.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Second implementation step of stage 1b, in tools/token-e2e/frozen_checks.py.

What: a key recipe for every graded template, computed from pinned Git content and
recorded as unknown with a reason when a source is absent (never guessed); the T1 heading
and three-way token-line counts, the T3 definition segment, T5/T11/T26 site analysis (Name
calls, attribute calls, imports, comment and string mentions, enclosing functions), the
T6 trust() segment, the T8 name sets with their distractors, the T9 frozen input, the T28
patch counts and the memory and catalog keys. Then strict payload decoding per TOON spec
v4.1.1 section 2 (CLI 4.1.1 `--decode`, never `--no-strict`; JSON-type identity so a
boolean never equals a number; ordered keys unless the reading says otherwise) with the
correction-6 candidates (fences, contiguous line blocks that start like a TOON header or a
JSON array or object, evidence strings, the whole answer; a prose `[1]: url` is not a
payload), and the class A oracles for the templates that need only an answer and a key.

Sources: repair-u9.design.md a4, R3, R4, R5, R13; TOON spec v4.1.1 SPEC.md section 2;
corrections items 5 and 6. Scanners are linear character scanners, not regular
expressions, so an answer cannot stall the grader; an answer is never dereferenced.

Tests now passing: F4_Keys (7), F5_SymbolSites (12), F6_StrictDecode (17), F32_TokenLines
(3), G_TemplateSweep (11), F18_LiteralAnswers (1): 51 tests, OK, in 22.9 s. Not yet built:
T0 and the captures, page facts, memory keys, the T14 query time, the builder and binding
checks, grade.py (spec, bind, keys, capture).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…emory

Third implementation step of stage 1b, in tools/token-e2e/frozen_checks.py.

What: the T0 oracle over six captures per arm, taken before and after the arm, under the
arm's conditions and plain (R10, U9-D11): a wrong or older subject fails, a missing capture
fails, a pre/post disagreement is unknown(capture_conflict), a field that differs between
the two captures (or every field when the Codex sandbox could not be verified, recheck
RV-06) is environment-dependent and an answer that disagrees is unknown(env_mismatch), and
an unparsed count is unknown until a D-extract quote is verbatim in the answer.

Correction 2: the unittest re-run is the arm's own command, `python3 -m unittest
tests.test_host_requests`, with PYTHONDONTWRITEBYTECODE=1, in a temporary copy of the tree
(isolate the tree, not the interpreter): `python3 -I -S` cannot import the tests package
(P0's control shows it), and a post capture is discarded, never re-run, when the tree
changed. Git calls set GIT_OPTIONAL_LOCKS=0 so observing a tree never writes its index.

Also: the T31 builder check (identity, base, porcelain with ignored files, bytes before any
execution, then greeting() in an isolated interpreter), the T36 binding check, correction 3
(the T14 query time is the first tool_use naming the whole word agentsview, in a command,
in ctx_execute code or as the MCP server), page facts from html.parser text with key drift
compared on facts, curl page captures, the R9 memory keys (records created before SINCE
holding the anchors; E_MEMORY_KEY per topic in sorted order), qmd coverage, the Linux
process listing and the exported-tree post-window runs (git archive, no write to the exec
checkout).

Sources: repair-u9.design.md R9, R10, R12, R13, a4; corrections 2 and 3; git-status(1)
porcelain v1; ai-memory 2.4.1 and curl help read on this host.

Tests now passing: F7_T0 (9), F7b_T0Rerun (1), F9_Builder (8), F10_Binding (6),
F31a_T14QueryTime (5), F33_PageFacts (5), F25a_MemoryKeys function tests (3): 37 tests, OK.
Not yet built: grade.py (spec, bind, keys, capture, grade's regeneration check).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Independent review of 857dc393 and 9496089a (a second loop of this agent, read from
committed bytes) found gaps; each is verified here by a test that fails first, or by a
disarmed-guard mutant where the guard already exists. Sources: that review (bytecode
caches, the copy, the plain value, R4 duplicate keys, the real `keys` path, the judge
effort rule relayed by the coordinator: the GPT-6 judge runs at max, never on codex_lane's
default `high`).

Failing at the current code (observed, host paths removed):
- F7_T0.test_bytecode_caches_are_not_a_tree_change: an ignored scripts/__pycache__ changes
  tree_state (status_sha256 193149b9... instead of the empty-status e3b0c442...).
- F7_T0.test_environment_dependent_skip_count_is_unknown (changed): the plain capture's
  value "skipped=0" gives ('pass', set()); expected ('unknown', {'env_mismatch'}). The
  arm's own capture is the key; the plain one only marks which fields are
  environment-dependent.
- F9_Builder.test_bytecode_caches_are_not_extra_changes: ('fail', {'extra_changes'}) for a
  builder tree whose only extra is fixtures/__pycache__/; expected ('pass', set()).
- F7b_T0Rerun.test_a_planted_bytecode_cache_is_never_executed: an unchecked-hash .pyc
  planted in tests/__pycache__ ran; facts {'ran': None, 'status': None, ...} instead of
  {'ran': 2, 'status': 'OK', ...}.
- F2_ExactTextRegistry.test_a_dropped_task_that_is_still_in_the_preregistration_is_refused:
  "expected refusal 'E_TEMPLATE_TASKS reason=dropped_present task=reuse-296-15', nothing
  was refused".
- F20_GradingBlock.test_a_judge_that_is_not_pinned_to_max_effort_is_refused and
  test_malformed_hex_fields_are_refused: no refusal line yet ('' != 'E_GRADING_BLOCK
  field=judges.claude_answers.effort' and '... field=tool.revision'); grade.py is a
  marker stub at this commit.
- F27c_KeysOnRealContent.test_keys_at_the_pinned_revision (new: `keys` over this
  repository's content at the pinned revision, a local clone plus one fixture commit) and
  F27b_CaptureCommand.test_post_arm_discards_a_changed_tree_and_lists_new_processes (now
  starts a uniquely named process, so other sleepers on the host cannot flake it): both
  end in FileNotFoundError for the missing grade.py outputs.

Passing already, and why it is honest to say so:
- F6_StrictDecode.test_duplicate_keys_and_non_finite_constants_are_refused and
  F19_Mutants.test_M25_duplicate_key_refusal_off: the guard exists (16ba45d5); M25 (the
  duplicate-key hook replaced by dict(pairs)) flips exactly that test.
- F6_StrictDecode.test_prose_that_looks_like_a_footnote_is_not_a_payload: the guard was
  written before this test; it covers `[1]: url` and `[2] a note` next to a real payload.
- F7b_T0Rerun.test_the_run_happens_in_a_copy_not_in_the_tree and
  F19_Mutants.test_M26_in_place_run: the copy exists (3c095eb5); M26 (the copy replaced by
  a link back to the tree) flips exactly the marker test, while the other F7b test stays
  green, which is why the old test alone did not prove the copy.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…view fixes)

Fixes for the loop-2 review, each driven by the failing tests of 709f17e4.

- Bytecode caches (review F-1 and section 6): a `__pycache__` component or a .pyc/.pyo file
  is neither part of the tree state nor an extra change, so a child that ran the unittest no
  longer makes every post capture `tree_changed` and a builder that imported before.py is no
  longer a false FAIL; every other ignored file still counts. The copy never holds bytecode
  caches, the re-run sets a fresh empty PYTHONPYCACHEPREFIX (a planted unchecked-hash .pyc
  runs even under `python3 -I -S`, and -I ignores the environment), and the T31 greeting is
  compiled from the verified retained bytes in an isolated `-I -S -B` interpreter instead of
  importing a file. This narrows the design's `--ignored=matching` and R13's `-I -S` on
  purpose; recorded as a deviation with the review's evidence.
- T0 key (F-3): the arm's own capture is the key; the plain capture only says which fields
  are environment-dependent, so the plain value alone is unknown(env_mismatch).
- Judge route (G-1, the coordinator's rule): the grading-block schema pins effort to max for
  both judges and freezes model, role and agent type as design f2 lists them; a block naming
  `high` is refused with its dotted field.
- Dropped task still present (G-2): E_TEMPLATE_TASKS reason=dropped_present task=<id>.
- Hex fields (G-5): tool.revision 40 hex, tool.sha256.* and registry_sha256 64 hex, refused
  by dotted field after the schema pass.
- Git wrappers (G-4): timeouts, a return-code check and an E_GIT refusal for a malformed
  cat-file stream. normalize() documents that both sides of a comparison must pass through it
  (G-3).

Tests now passing: F7_T0 (11), F7b_T0Rerun (3), F9_Builder (9), F2 (5), F20b (4) and
F19 M26: 32 tests, OK.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Both are defects in test data, not in an assertion's intent.

1. F25a test_since_is_the_earliest_window_start built its fixture repository without
   recipes/host-request-lane.md, so the T7 key was unknown(key_missing) and its
   memory_records could not be read from the key. The fixture now carries the recipe.
2. F27b test_post_window_runs_the_acceptance_commands_in_an_exported_tree called
   make_bindings a second time into the same output path, which the create-only rule
   correctly refused (E_PATH reason=exists). make_bindings takes an out_name.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The command line of the grader, in tools/token-e2e/grade.py, on the kernel of
frozen_checks.py.

- spec: reads the preregistration blob at a 40-hex commit, verifies its sha256 against the
  newest "Amendment N seal" table of the README at that commit (no override flag: argparse
  refuses one as E_ARGS field=usage), requires `## Amendment 4` and the `grading` block,
  validates it (schema, hex fields, registry hash, pinned judge route), builds the inventory
  and the graded task array and writes canonical JSON create-only, mode 0600.
- grade: regenerates the spec from its own commit and refuses a spec whose bytes differ
  (E_SPEC_MISMATCH). The grading itself is the next stage, so a matching spec ends in
  E_STAGE reason=grade_not_built (design deviation: `grade` takes --repo, which the b3 table
  omits and R20's regeneration needs).
- bind: the private launch inputs (per-arm worktree paths and bases, retained inputs with
  their sha256, sentinels with their hashes, both windows, roots, anchors), refusing unknown
  keys, missing values and hash mismatches with E_BIND field=<f>; the run token is kept only
  as a sha256; U10's codex-bindings.json merges by task (a fixture-shaped consumer until U10
  merges); roots gain exec_checkout (R13 inventories it; the design's bind contents omit it).
- keys: every freeze key from pinned Git content; --memory freezes the historical records
  before the EARLIEST SINCE of either window, so a page written between the windows is not
  history for a Codex task (U9-D13); E_MEMORY_KEY refuses before anything is written.
- capture: w-open/w-close curl the four frozen pages and record facts; pre-arm/post-arm take
  the six T0 identities twice per tree (the arm's conditions and plain), inventory the exec
  checkout with the porcelain options, discard a changed tree's post capture, list processes
  started since the pre-arm capture and key T11 clones; post-w runs the acceptance commands
  and py_compile in a git-archive export. RUN_TOKEN must match the bind's hash.

Sources: repair-u9.design.md b3, R10, R12, R13, R19, R20, R22 and the corrections; ai-memory,
qmd, codex and curl native help read on this host.

Tests now passing: the complete tests.test_token_e2e_grader suite, 164 tests, OK, 0 skipped,
in 82 s (Python 3.13.15, toon 4.1.1, git 2.43).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… tools

Two changes that keep the suite cheap and portable, found while checking how CI runs it
(`python3 -m unittest` over everything, on Linux and macOS runners).

- Speed: `keys` for all 75 tasks took about 20 s because every symbol re-walked all 240
  scripts and tests. A file that never spells a symbol has no site for it, so only files
  that do are parsed, once per source object, with the token scan and the enclosing-name
  map built on demand. All 75 keys now take 1.9 s with identical values (the same four
  Name-call sites, 16 mentions and 20 subprocess.run calls); the whole suite went from
  82 s to 24 s.
- Portability: the design says CI may skip the TOON and private tests while the workstation
  acceptance run shows 0 skips. The tests now skip, not fail, when the TOON CLI or node is
  absent or Linux /proc is missing (the process listing), and P0 is split into
  python-interfaces, node-exports and toon-version tests so one missing tool no longer
  hides the others. Verified on a PATH without toon and node: 166 tests OK (skipped=11);
  on this workstation: 166 tests OK, 0 skipped.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… page URLs

Two small hardenings found on a final read of the command line.

- bind: a launch-args file whose own `arm` differs from the ARM= label it is bound to is
  refused (E_BIND field=arm); failing first: "'' != 'E_BIND field=arm'".
- capture_page: a URL that is not http or https is never handed to curl (it could parse as
  an option); the record is {status: None, error: bad_url}. The guard was written before
  its test, so the test passes on first run; it asserts curl was not run at all.

Tests: F26_Bind (11) and F27b_CaptureCommand (9) OK.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Second independent review round (loop 2, from committed bytes of 16ba45d5, 3c095eb5 and
7defe604): every finding below was reproduced by running the committed code, and each
is covered by a test that fails first. Sources: that review (REVIEW-1b notes H-1..H-5,
J-3..J-6, K-1, K-2) and the design's R3 ("FAIL means demonstrably wrong; an absent key
fact is unparsed") and F15/R21 (a deterministic FAIL sends no packet to the semantic judge).

The rule the tests pin (H-1): a single-valued fact passes only when every recognised value
equals the key; when the values disagree and one equals the key (a hedge such as "9 or 10",
"96 (or 66)", "yes or no", "exit code 1 or exit code 0") it is unknown(unparsed), so the
D-extract fallback may decide and the lower bound never rises; when none equals the key it
fails. An absent key fact (a paraphrase) is unknown(unparsed), not a FAIL, so the judge
still sees it; an absent required citation or literal still fails.

Failing at the current code (observed, host paths removed):
  F20_GradingBlock.test_a_memory_query_that_could_parse_as_an_option_is_refused: '' != 'E_GRADING_BLOCK field=memory.reuse-296-07.query'
  F20_GradingBlock.test_a_page_url_that_could_read_a_local_file_is_refused: '' != 'E_GRADING_BLOCK field=pages.json'
  F26_Bind.test_codex_bindings_are_merged_and_checked: KeyError: 'arm'
  F26_Bind.test_codex_trees_are_matched_by_arm: '/one' != '/two'
  F26_Bind.test_run_record_args_with_another_token_or_arm_conflict_for_every_task: Lists differ: [] != ['reuse-296-00', 'reuse-296-02']
  F27b_CaptureCommand.test_a_structurally_broken_spec_is_an_internal_refusal_not_a_traceback: 'Traceback (most recent call last):' != 'E_INTERNAL stage=capture'
  F27b_CaptureCommand.test_curl_is_restricted_to_http_protocols: ValueError: '--proto' is not in list
  F27b_CaptureCommand.test_post_arm_retains_the_builder_bytes_and_entries: KeyError: 'builders'
  F27b_CaptureCommand.test_the_t0_commands_never_see_the_operator_environment: 'FAILED' != 'OK'
  F6_StrictDecode.test_a_toon_cli_that_is_not_4_1_1_is_refused: 'toon_strict_decode' != 'E_TOOL tool=toon reason=version'
  F7_T0.test_a_hedged_skip_count_is_unparsed: ('pass', set()) != ('unknown', {'unparsed'})
  F7_T0.test_the_first_subject_mentioned_must_be_the_newest: ('pass', set()) != ('fail', {'subject_not_newest'})
  H_OracleReviewFindings.test_a_catalog_answer_with_the_citation_but_paraphrased_facts_is_unparsed: ('fail', {'fact_missing', 'anti_pattern_missing'}) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_a_hedged_payload_sum: ('pass', set()) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_a_paraphrase_of_the_release_pin_reaches_the_judge_instead_of_failing: ('fail', {'fact_missing'}) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_an_absent_scope_name_is_unparsed_because_the_judge_settles_wording: ('fail', {'scopes_missing'}) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_t1_heading_count_is_not_satisfied_by_numbered_heading_lines: ('pass', set()) != ('fail', {'heading_count'})
  H_OracleReviewFindings.test_t1_token_lines: ('pass', set()) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_t27_exit_code: ('pass', set()) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_t28_numbers: ('pass', set()) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_t32_verdict_and_latency: ('pass', set()) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_t34_count: ('pass', set()) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_t35_first_and_last: ('pass', set()) != ('unknown', {'unparsed'})
  H_OracleReviewFindings.test_t8_a_stated_count_must_equal_the_number_of_names: ('pass', set()) != ('fail', {'count'})

Ran 191 tests: failures=21, errors=3 (all of the 24 above). Environment: Python 3.13.15,
toon 4.1.1.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Implements the fixes for the loop-2 findings that 73ed708e covers with failing tests
(H-1..H-5, J-3..J-6, K-1, K-2), in tools/token-e2e/frozen_checks.py and grade.py.

Correctness (R3): a single-valued fact now passes only when every recognised value equals
the key. A hedge that contains the key ("9 or 10", "96 (or 66)", "First 64 or 65",
"Verdict: yes or no", "exit code 1 or exit code 0", "Latency sum: 124 or 999") is
unknown(unparsed), so the D-extract fallback may decide and the lower bound never rises;
no equal value fails. Alternatives joined by "or", "/" or "vs" are recognised in either
direction inside one clause. T34 reads a bare integer, else integers beside a count word,
else every integer; T1's heading count reads only the plural label, so "Heading 10:" lines
no longer satisfy it; T8 checks a stated function count; T0 requires that the first subject
the answer names is the newest. An absent key fact (a paraphrase of the release pin, the
catalog facts, a scope name) is unknown(unparsed), not a FAIL, because a deterministic FAIL
sends no packet to the semantic judge; an absent required citation still fails.

Safety: the TOON CLI on PATH must be 4.1.1 before any decode (E_TOOL tool=toon
reason=version) and argv keeps the bare name; curl is restricted to http and https and a
page URL or memory query that could parse as a file scheme or an option is refused at spec
time; the six T0 identities, the unittest that runs the tree's code, the greeting run and
the post-window runs get an allowlisted environment (PATH, LANG, TMPDIR and a HOME that is
not the operator's), so no operator secret nor RUN_TOKEN reaches them; an unexpected
exception is one `E_INTERNAL stage=<command>` line with exit 2 instead of a traceback with a
host path and exit 1 (GRADE_DEBUG=1 adds the traceback after it); binding_conflicts
also compares the run token and the arm; the post-arm capture keeps each builder tree's
porcelain entries and before.py; Codex trees are matched by arm and an ambiguous task is
refused (E_CAPTURE field=codex_tree).

Sources: the loop-2 review notes; repair-u9.design.md R3, R19, R22, F15/R21; TOON spec
v4.1.1; the real ai-memory CLI accepts `--` before a positional query (checked).

Tests: the complete tests.test_token_e2e_grader suite, 194 tests, OK, 0 skipped; three new
disarmed-guard mutants (M27 a hedge that contains the key passes, M28 the version pin off,
M29 the operator environment reaches the commands) each flip exactly their tests.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Coverage the stage lacked, found while preparing the handoff.

- F25c_QmdCoverage: `keys --qmd` was implemented but untested. A fake qmd 2.8.3 prints
  listing lines that end in qmd://<collection>/<file>; the coverage of adoption/update.md is
  read from them, an unreadable index is recorded as not queried (never guessed), and the
  keys command puts the record into the T4 key. The tests passed on first run because the
  code existed; M30 (coverage never found) flips exactly the two that assert a hit.
- F27c: the spec over this repository's real sealed preregistration bytes (with a fixture
  Amendment 4 grading block, since the sealed bytes have none) is written twice and compared
  byte for byte. The pipeline's acceptance item 3 asks for `grade.py spec` twice with the
  sealed hash "as the expected value"; the design forbids any expected-hash flag (R20, F20)
  and the sealed bytes carry no grading block, so `spec` at c0966da refuses with
  E_GRADING_BLOCK field=missing (design acceptance 4); this test is what can be checked.

Tests: 198 tests in the complete tests.test_token_e2e_grader suite, OK, 0 skipped.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Closes the last stage-1 gap found while mapping every review and recheck finding to a
disposition (recheck new defect 13: a tracked file that an earlier arm's child modified or
deleted in the shared exec checkout changes the same keys as a new file does).

The pre-arm exec-checkout inventory already lists untracked and modified entries. The test
now also deletes a tracked file and expects ("D", path) beside ("??", path) and ("M", path).
porcelain_entries runs `git status --porcelain=v1 -z --untracked-files=all --ignored=matching`
(design R13), so the code needed no change and the test passed on its first run: it is a
coverage addition, not a failing-first guard.

Stage 1 of the U9 frozen-check grader is complete at this commit (registry, spec, bind, keys,
capture and the pure oracles; corrections 2 and 3 applied): tests.test_token_e2e_grader ran
198 tests, OK, 0 skipped, at the parent. Stages 2 and 3 (evidence and grading; judges,
controls, export and docs) build on it.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Found in the final read of the capture conditions. The T0 identities (six commands, and the
unittest that runs the tree's tests in a copy) get an allowlisted environment with a
throwaway HOME (review J-5, K-3). That HOME also hides the operator's per-user Python
packages. tests.test_host_requests skips one test unless PyYAML is importable, so on a host
where PyYAML sits in the user site the child (real HOME) reports skipped=0 while both grader
captures (arm conditions and plain, same allowlist) report skipped=1. The skip count is then
not marked environment-dependent, and a correct answer fails T0. Correction 2 says the arm's
own environment; the allowlist must keep the user base visible and nothing else.

On this workstation PyYAML is not installed anywhere, so the two runs already agree here
(49 tests, OK, skipped=1 in both, checked in place versus the allowlisted capture); the
defect needs a host with a user-site package, which the tests build: a fake per-user base
holding one module (u9userpkg), a tree test that skips unless it imports.

Failing at the current code (observed):
  F7b_T0Rerun.test_a_user_site_package_stays_visible_under_the_allowlisted_environment:
    {'ran': 1, 'status': 'OK', 'skipped': 1, ...} != {'ran': 1, 'status': 'OK', 'skipped': 0, ...}
  F27b_CaptureCommand.test_both_t0_conditions_keep_the_operators_user_site_packages:
    the same facts through `capture --phase pre-arm`, arm and plain conditions.
Passing at this parent by design: F19_Mutants.test_M31_user_base_dropped_from_the_allowlist
(the mutant is the current allowlist, so it flips exactly the two tests above and leaves the
operator-environment control green).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Fixes the defect the tests of 9e2453a5 fail on: the allowlisted environment of every grader
subprocess (minimal_env) replaced HOME with a throwaway directory, which hid the operator's
per-user Python packages from the T0 runs. tests.test_host_requests skips one test unless
PyYAML imports, so on a host with PyYAML in the user site the child's skip count differed from
both grader captures, and a correct answer failed T0 without being marked environment-dependent.

minimal_env now also sets PYTHONUSERBASE to the base the operator's own interpreter uses (the
variable when set, else site.getuserbase()). It is a package directory, not a secret; every
other operator variable, RUN_TOKEN included, stays out (F27b's operator-environment test and
mutant M29 still hold). Sources: correction 2 ("the arm's environment"), review J-5 and K-3
(the allowlist), Python site documentation for PYTHONUSERBASE and getuserbase().

Measured before the change on this workstation, where PyYAML is not installed anywhere:
tests.test_host_requests in place with the operator shell gives {ran 49, OK, skipped 1} and the
allowlisted capture gives the same facts, so the recorded T0 keys of this host do not change.

Tests: F7b_T0Rerun, F27b_CaptureCommand and the M29 and M31 mutants, 20 tests, OK; the two
tests of 9e2453a5 now pass and M31 (the old allowlist) flips exactly them.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Stage 2a, group 1. Design repair-u9 R1 (attempt classes), R2 (answer carriers),
R6 (task, blind and control rules), R15 (M8), R16 (G-Q), R17 (G-C denominators),
tests F11-F14 with the D1 matrix (26 cases, mutants MUT-A..E) and the binding
corrections 1 (Agent-tool frame and trailer, observed on 33 of 33 foreground
results of this host's client: a frame line, every body line indented by two
spaces, an agentId line, a usage block) and 4 (a superseded run is classified
by its own transcript's final row; a run that ended with an answer or a
task-caused failure is a graded attempt).

Cross-check outcomes are kept apart (advisor note on the runner log): the
logged response equal, differing (carrier_mismatch) or unavailable (absent or
unparseable, never a mismatch).

Observed at the parent: 52 tests, 118 errors counting subtests, none passing.
Each fails with AttributeError: module 'evidence' has no attribute '<name>'
(workflow_attempts 26, decide_task 57, codex_exec_attempt 11, agent_child_attempt
7, follow_persisted 4, attempt_effect 2, blind_effect 2, g_c_denominators 2,
strip_agent_result 1; g_q and m8_lane fail the same way). There is no
tools/token-e2e/evidence.py at the parent, so the name resolves to the
repository's evidence/ namespace directory and the error is an AttributeError
instead of a ModuleNotFoundError.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Stage 2a, group 2. Design repair-u9 R7 (M12: five tests per blind attempt,
join through the U4 ledger row and the U2 run-mode child, strict transcripts
through the kernel bridge, the U4 m12_inputs cross-check, status), R8
(retrieval of each frozen URL by the child itself, in the family window) and
R9 (a historical hit counts only when it resolves to a record frozen before
SINCE); tests F22, F24, F25b. Each contamination case names its decided and
alternative reading (R2-11..R2-14, R2-18).

Sibling shapes are fixtures of the stated ones: U2 callLedger records
({session_id, owner, tool_use_id, tool, server, state, cause, native_status,
background}) and run-mode children (agent_id, label, lanes.measurement.
hook_context.inserted and by_hook), U4 join-ledger rows. measureTranscript is
the one kernel function that exists at this base, so the strict-transcript
test runs the real one through the bridge.

Observed at the parent: 39 tests, 39 errors, none passing, each
AttributeError: module 'evidence' has no attribute '<name>' (m12_attempt 15,
claude_calls 18, codex_calls 2, kernel_hook_context 1, m12_hook_source 1,
m12_cross_check 1, m12_status 1; retrieval_check and memory_check fail the
same way). The name resolves to the repository's evidence/ namespace
directory because tools/token-e2e/evidence.py does not exist yet.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Stage 2a, group 3. Design repair-u9 R14 (M7: seeded encodes, strict round
trips, ineligible encodes, thresholds), R11 (T14 reported sessions, query
time, owned survivors), R13 (tree drift), R2-17 (T2 recovery), T31 graded from
the post-arm capture, and the binding corrections 5 (toon --stats prints the
document on stdout and two status lines on stderr, both merged by Bash, ctx
and Codex results; the lines are dropped by their documented shape, read from
the installed CLI 4.1.1 source) and 6 (D-extract values join the payload
candidates); tests F28, F31b, F34, F9b, F4b.

Four defects of the stage-1 build are found here and each has its test:
- T27 with a recorded acceptance run raises NameError (`summary` is not a
  name in oracle_T27; it must read key["summary"]).
- capture files are named arm-<ARM>-<phase>, so the Claude and Codex arm B
  collide in one directory; the tests name them arm-<family>-<ARM>-<phase>.
  The seven existing F27b tests that read the old names are updated.
- list_processes records no pid, so a survivor's descendants cannot be found.
- the spec carries no thresholds; R14 reads /thresholds/M7/criteria and M8
  needs the 0.8 rate and the minimum of five attempts, all from the sealed
  bytes.

Observed at the parent: 63 tests, 1 failure and 62 errors, none passing.
- 50 tests: AttributeError: module 'evidence' has no attribute '<name>'
  (apply_tree_drift 10, claude_calls 9, t14_check 9, isolate_toon_document 7,
  m7 7, toon_facts 4, grade_builder_capture 4, builder_observed_tree,
  command_programs, owned_survivors, recovery_check).
- NameError: name 'summary' is not defined (oracle_T27).
- KeyError: 'thresholds' (spec) and KeyError: 'pid' (list_processes).
- The six renamed F27b tests: FileNotFoundError for arm-claude-B-*.json.
- test_arm_captures_are_named_by_family...: AssertionError: 2 != 0 :
  E_PATH reason=exists (the second family's capture is refused).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Stage 2a, group 4. Design repair-u9 R19 (identity: rows from started labels,
launch records and U10's Codex rows, guesswork refused by kind, arm-T rows
accepted as reported-only, the validator's code surfaced as
E_IDENTITY_INVALID), R22 (private side: create-only, work-tree refusal, the
canary over keys, shapes and gathered values), F16 (grade twice then regrade
give identical bytes and call no model), F17 (privacy), correction 9 (M11 for
Codex role children: role_child_state with the four outputs of U13's reference
specification, arm counts) and R2 codex_subagent (child rollout copy, the
seed-binding-5 boundary), plus the stage-2 mutants of F19 (M4, M5, M14-M17,
M18b, M19b, M20, M32 superseded-as-inadmissible, M33 trailer not removed,
M34 whole merged result decoded, M36 drift ignored), each asserting the exact
set of tests it flips.

`grade` needs its inputs after the spec check, so the stage-1 expectation of
E_STAGE reason=grade_not_built is replaced by E_ARGS field=bindings
(test_the_grade_needs_its_inputs_after_the_spec_check); the two F21 tests keep
passing because the regeneration check runs first.

Observed at the parent: 56 tests, 39 failures and 26 errors (subtests counted),
the two F21 tests passing.
- CLI tests: AssertionError: 'E_ARGS field=usage' != 'E_IDENTITY_SOURCE
  kind=<main|agent|team|workflow|run_token|codex_rows>' and != 'E_IDENTITY_INVALID
  code=E_ROW'; AssertionError: 2 != 0 (the grade and regrade commands do not
  exist); 'E_STAGE reason=grade_not_built' != 'E_ARGS field=bindings'.
- AttributeError: module 'evidence' has no attribute 'role_child_state' (8),
  'codex_subagent_attempt' (3), 'assert_no_private', 'm11_summary',
  'parse_role_toml'; the mutant tests fail on the missing attribute they patch.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
evidence.py part 1 and 2 and node_bridge.mjs. Design repair-u9 R1 (attempt
classes from launch-level signals), R2 (workflow_child, codex_exec,
agent_child and transcript carriers; the Agent tool_result frame and trailer
removed structurally, correction 1; persisted output followed only under the
declared roots), R6 (task, blind and control outcomes through attempt_effect
and blind_effect), R15-R17 (M8, G-Q, G-C), R7 (M12: five tests, the join
through the U4 ledger row and the U2 run-mode child, the m12_inputs
cross-check), R8 (retrieval by the child itself, in the window), R9 (a hit
resolves to a frozen record) and R2-17 (T2 recovery evidence).

Superseded runs are classified by their own transcript's final row
(correction 4): a rate limit or a login failure before any model turn is
inadmissible, an answer or a task-caused failure is a completed attempt, a run
that only stopped is an interrupted driver. The runner's logged response is a
three-outcome cross-check (equal, mismatch, unavailable): an unparseable or
missing log is never a mismatch.

node_bridge.mjs imports the kernel as a namespace and reports a name that a
sibling has not merged (U1 loadShellParser, shellParserStatus, commandInvocations;
U2 callLedger, m14State; U4 validateIdentityTable) as unavailable; at this base
only measureTranscript is real, and F22 runs the real one on a strict transcript.

Also fixes one fixture defect of the group-1 tests found here: the wait-notice
subtest built two worlds in one directory.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
evidence.py parts 3-6, grade.py identity, grade and regrade, and the stage-1
gaps found while building stage 2. Design repair-u9 R14 (M7: the seeded CLI
encode, strict round trips and ineligible encodes; toon --stats merges the
document on stdout with the two status lines on stderr, which are dropped by
their documented shape before the strict decode, correction 5, and a D-extract
payload joins the candidates only with verbatim quotes, correction 6), R11
(T14 reported sessions against the table rows that started before the first
archive query, and an owned-survivor filter: started inside the child's
lifetime and named like a program it ran, or a child of such a process; other
in-lifetime processes are counted apart and never fail the child), R13 (tree
drift: modified, deleted, untracked and ignored entries in a key scope turn a
FAIL into unknown; T31 graded from the post-arm capture, running the key's own
after.py bytes), R19 (identity: rows from journals, launch records and U10's
Codex rows; a local twin of U4's R5 validation that accepts arm-T rows as
reported-only; guesswork refused by kind), R22 (canary over keys, shapes and
gathered values; create-only private writes), R16-R18 and correction 9
(role_child_state).

grade collects one private record per run and evaluates them as a pure
function; regrade reads only the private directory, so the aggregate and
grades.jsonl are byte-identical and no model or launcher is ever called.

Fixes to the stage-1 build, each with a failing-first test in the group-3 and
group-4 commits:
- oracle_T27 read an undefined `summary` when an acceptance run was recorded.
- capture files are named arm-<family>-<ARM>-<phase>: the Claude and Codex
  arm B collided in one directory.
- the spec carries the frozen thresholds; list_processes records the pid;
  Refusal takes its code positionally so a field may be named `code`.
- keys.json is no longer key-sorted: canonical form lost the field order of the
  table records that R2-02 compares (found by the F16 end-to-end run).
The positive control is measured only for its Read rows, never as blind
evidence (its own hook row is what qualifies it).

Two fixture defects of the earlier test commits are fixed here (subTest ids
in failing_with, two rollout copies in one directory).

Full module: 396 tests OK, 0 skipped, on the workstation.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Observed on this host's real Codex rollouts (120 rollouts, counts only): MCP
calls are recorded as event_msg item_completed items of type McpToolCall with
a server field, and never as function calls named mcp__<server>__<tool>; the
turn_context payload carries model, effort, sandbox_policy and cwd as the M11
grader reads them. role_child_state read only the exec-events spelling
(mcp_tool_call), so a child that called a server outside the parent's set
passed the bindings check.

Also adds two P0 checks: the bridge reports which kernel names exist at this
base, and the real summarizeRun output has the run-mode child shape the grader
consumes (agent_id, label, lanes.measurement.hook_context.inserted and by_hook).

Observed at the parent: test_an_mcp_item_in_the_rollout_shape_is_read_too fails
with (True, []) != (False, ['bindings']).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Design c1: the private row records descriptive fields beside the grade: the
payload bytes (T9, T16-T20), the main-transcript bytes (T34), the conversion
state and bytes with the scope terms it lost (T13) and the T14 reported and
expected counts (RV-30, ledger item). They decide nothing.

Observed at the parent: KeyError: 'recorded' (two tests: the mini-run rows have
no recorded object), and ('pass', {}) != ('pass', {'reported': 1, 'expected': 2})
for the T14 counts.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Makes the two failing-first commits before it pass. role_child_state reads an
MCP call as an item_completed McpToolCall item (the spelling of this client's
rollouts, observed on real ones) as well as the exec-events spelling. The
private rows gain the descriptive recorded object of design c1: payload bytes
(T9, T16-T20), main-transcript bytes (T34), the conversion call's tool, state
and bytes with the scope terms its output lost (T13), and the T14 reported and
expected counts, which t14_check now returns as the detail of its result.

Real-data checks (aggregate scans on this host, counts only, not committed):
- agent_child_attempt over 350 real Agent-tool calls: 36 foreground results all
  pass with the inline carrier (the frame and trailer removed structurally), the
  274 asynchronous launches fail not_inline, 20 children were not found, 19
  ended inadmissible (11 usage_limit, 8 interrupted_driver), 1 api_error.
- workflow_attempts over 934 labels of 120 real Workflow runs: no exception
  (879 carry another workflow's response schema, hence schema_invalid).
- m12_attempt over the 17 real blind transcripts: the attachment kinds before
  the first answer are exactly the frozen allowlist, the 15 Agent-tool ones fail
  the first-prompt and marker tests (the routing marker), the 2 Workflow ones pass.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Correction 9 (M11 for Codex role children) is wired end to end: the role file
at exec_rev, the parent's effective server set from the bindings and the
parent's rollout copy feed role_child_state, and the aggregate publishes the
per-arm counts with their unknowns. `bind` must keep the parent server set of
U10's bindings file (an assumed field: U10's codex-bindings.json is not merged).

Also covers the judgments path that stage 3 will feed: a judgment that holds
passes the D component, one that does not fails it, an unavailable one leaves it
unknown with its reason, and `regrade --judgments` equals a fresh grade with the
same judgments. Two more mutants: M37 (judgments ignored) and M38 (the role
child check never runs).

Observed at the parent: KeyError: 'parent_servers' (bind drops the field), and
the role-child counts come out {A: unknown 1, B: unknown 1} instead of {A: fail 1,
B: pass 1}. The judgments tests already pass (their guards exist); M37 and M38
give the condition-absent evidence once the wiring is in.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Makes the role-child wiring tests of the previous commit pass: `bind` keeps the
parent_servers list of U10's bindings file (refusing a non-list) so
role_child_state can compare a role child's MCP servers with the parent's
effective set. The field name is an assumption until U10's file merges.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
scripts/validate.py scans tracked files for private-content shapes and failed on
two literals of the F17 canary test: a UUID-shaped value and a Windows user
path. The test still needs both shapes to prove the shape scans, so it now builds
them from parts, and no committed line has either. validate.py exits 0 again
(status passed, 8083 hashed files).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Source: the advisor's second review of stage 2, plus two checks of my own
against U10 (its design section 3.1 and codex_launch.copy_rollouts keep the
rollout copies in attempts/<identity>/rollouts/, not beside them) and the
retained exec streams (command_execution items are observed there, 4 of 4
completed, fields aggregated_output, command, exit_code, id, status, type).
The test fixtures move to the U10 layout; MiniRun.codex_exec now also writes the
thread id of U10's row, command items and an optional rollout copy.

Failing at 28d4604f, first differing value:
- role children: m11_summary KeyError 'not_applicable'; the rollouts under
  rollouts/ are not found, so m11_roles is {} where the child count is expected
- M12: status keyed on the correctness of the answers gives ('fail',
  'blind_task_without_clean_verdict') for five clean wrong verdicts; the sums
  count rows 2 and hook_rows 1 for a superseded run beside its final run
- a call with no ledger record graded as a failure: memory ('fail',
  no_historical_hit) not ('unknown', call_state_unknown); the same for recovery,
  a toon encode ('not_encoded'), a child decode ([]) and a small encode (0, 0)
- M7: a Codex seeded attempt gives (0, 0, 0) not (1, 1, 0); a T9 answer payload
  gives (0, 0, 0) not (0, 1, 1); the wrapper encode is ineligible under both
  R2-01 readings; six natural payloads counted where five carry a payload
- T14: a count with no parsed identity fails count_mismatch not unknown; a Codex
  attempt fails no_archive_query where its rollout copy times the query;
  t14_check has no q_status or survival_observed, t14_facts and
  codex_archive_query_time do not exist
- an exec mcp_tool_call with status completed and a non-null error: 'succeeded'
- identity: a dedicated main session that called the Agent tool once is refused
  (E_IDENTITY_SOURCE kind=main)
- M8: (O, R) = (5, 0) where the join ledger gives (6, 2); regrade loses the ledger
  (FileNotFoundError on join-claude.jsonl)
- bridge validate_identity: (True, False) not (True, None) for an exception the
  validator did not code; validator_state does not exist
- a dotted mcporter call (ai-memory.memory_query) is not read as an ai-memory call

Passing at HEAD as controls: the retained exec item spellings, a Codex command
item encoding like a Claude call, a seeded attempt encoding only the seeded array.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Scout and others added 27 commits September 29, 2026 18:58
Stage 3a of the U9 frozen-check grader (design repair-u9.design.md R21 and
R22; tests F15, F19 stage-3 mutants, F23, F29, F30, F35; binding corrections
7 and 10). 81 tests are appended to tests/test_token_e2e_grader.py: F30
judge builders (strict-mode schema fixed point, Workflow syntax check, the
blind-lane-reviewer role bound on every agent call, claude-args and
collect), F23 web-table clauses and calibration files, F35 packets,
scrubbing, routes, isolation, timing and privacy (with the exact argv
element model_reasoning_effort=max of correction 10), F15 the judge contract
(judge_quote, judge_leak, judge_calibration, judge_refuted, retry cap,
usage-limit pause and resume, extraction), F29 export and check-html, the
`controls` and `differential` commands, and 14 disarmed-guard mutants. The
judge routes run against a scripted fake `codex` written by the test (the
precedent is tests/test_codex_lane.py); no model is ever called.

Observed at the parent 8766a39a (git stash of nothing; base suite 468 tests
OK): 80 of the 81 fail, 1 passes by design.
- 54 tests: ModuleNotFoundError: No module named 'judge'.
- 5 tests: FileNotFoundError for tools/token-e2e/frozen-check-judge.js,
  judge-template.md and calibration/oracle-controls.json (the syntax check
  reports ENOENT from node); 1 test: 0 calibration files, expected >= 22.
- 9 tests: 2 != 0 or 2 != 1 with E_ARGS field=usage from the missing
  subcommands export, check-html, controls, differential.
- 13 refusal checks: 'E_ARGS field=usage' != 'E_EXPORT_DIR reason=exists',
  'E_EXPORT_DIR reason=canary', 'E_PRIVACY' (10 including seven subtests)
  and 'E_DIFFERENTIAL field=inputs'.
- grade context has no `command`: KeyError: 'command'.
- Mutants: AttributeError for grade.export_dir_issue and grade.html_values;
  M17c and M51 fail on the expected-set comparison because their base test
  fails at the parent with or without the mutant.
- Passes at the parent by design: F23 test_the_first_clause_of_every_web_table
  _check_is_class_d (the five clauses were registered as class D in stage 1).

Helper changes: make_bindings gains `windows=`, MiniRun gains `windows=` and
`files=` (defaults unchanged, the 468 stage-1 and stage-2 tests still pass).
One lapse: three small expectation fixes made before this commit used a Bash
Python replace instead of Edit; the content equals an Edit.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Stage 3a, second commit (design e8 and correction 7, repair-u9.design.md R21
and b5 F30, F35). Five tests for `grade.py judge rehearse`: two planted
controls (one known pass, one known fail) through a route before Amendment 4
and before any window, with no bindings and no packets index. The codex
route prints a receipt of counts, versions and booleans and exits 1 when the
judge holds the known fail; a missing native credential is an isolation
refusal before any call; an existing rehearsal directory is refused; the
claude route prepares two Workflow items and later collects a clean run. The
real routes need Codex capacity and a Workflow run, so acceptance e8 stays
open: these tests run against the scripted fake `codex` only.

Observed at the parent 8766a39a plus the previous test commit: all 5 fail,
'E_ARGS field=usage' for the missing subcommand (2 != 0, 2 != 1, and
'E_ARGS field=usage' != 'E_JUDGE_ISOLATION check=codex_home' or
'E_PATH reason=exists').

Expectation corrections found while building stage 3b, none of them loosening
a guard (each was checked against the implementation before this commit; the
tests that changed passed for a wrong reason or asserted the wrong function):
- The fake codex ignored the quota probe only when `app-server` came first
  in argv; scripts/codex_quota.py runs `codex -c ... app-server`, so any
  position is now treated as the probe.
- test_collect_refuses_a_result_that_consumed_another_prompt_or_packet_hash
  built two worlds in one test (the second commit of the fixture repo
  fails); it is now two tests.
- M7 (quote verification off) no longer expects the two extraction-quote
  tests to flip: an extraction quote has a second guard, its mapping back to
  the raw answer text, so it stays refused with the first disarmed.
- M40 (calibration off) no longer expects the extraction-control test; the
  extraction calibration has its own function, and M40b disarms it.
- Judge test fixtures: JudgeWorld gains `embedded=` for a payload that no
  deterministic candidate detects.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The static files of the class D judges (design repair-u9.design.md R21 and
b1) and the frozen calibration data written before Amendment 4.
- judgment.schema.json: the judgment ({clauses[{id, holds, answer_quote,
  source_quote}], extractions[{component, values, answer_quotes}], leak,
  leak_text}), every field typed and every object closed with required equal
  to its properties, so codex_lane.strict_output_schema leaves it unchanged
  (checked: a fixed point) and Codex strict structured output accepts it.
- judge-template.md: the judge section and, after `<!-- refuter -->`, the
  refuter section, each with one {PACKET_BLOCK}; the packet is inline for the
  Codex route and a `Packet file:` line for the Claude route.
- frozen-check-judge.js: a saved Workflow in the pattern of
  tools/sota-convergence/adjudication-lane.js. One blind-lane-reviewer judge
  per packet at model opus and effort max, then one refuter per passing
  judgment (passes_only); no evidence-reviewer (its carrier block would void
  the judgment, correction 7). check-syntax.mjs prints SYNTAX_OK; the script
  holds exactly two agent calls, each naming the role, model and effort.
- calibration/T*.json: 21 files (the 19 class D templates, and T0 and T9 for
  extraction only) with clauses equal to the registry's, one reference-correct
  control built from a source excerpt, one paraphrased-correct control and two
  planted wrong controls per clause template, one that breaks each clause of a
  two-clause template; the five web-table files hold the frozen contradictions
  ("ensure_ascii defaults to False", "allow_nan=false emits NaN", "sort_keys
  sorts arrays", "raises TypeError", "loads accepts only str"); T0, T9 and the
  web templates carry two extraction controls each. Excerpts marked verbatim
  in the authoring are the pinned exec revision's own lines (trust(), the
  opening hunk, before.py and after.py, the release-pin lines, the catalog
  lines and anti-pattern cells).
- calibration/oracle-controls.json: 116 planted controls for acceptance e5
  (79 class A across 30 templates, 20 class B for T31, T14, T0 and T27, 17 class
  C for the answer carriers), each authored with the design's intent and
  checked against the real oracle before it was frozen.
Fixtures carry placeholder ids only (tok7fixture, fx9); no UUID, home path or
tool-call id shape.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
judge.py (design R21; corrections 7 and 10; sources tools/sota-convergence
codex_lane.py build_command, ISOLATION_ARGS, isolated_codex_home, child_env,
blind_path_issue, blind_child_argv, blind_audit and transcript_audit.py audit,
all reused unchanged, and adjudication-lane.js and adjudicate.py as the
pattern for a saved blind Workflow with a collect step):
- Packets: one per graded record whose class D clause is pending (not one that
  fails deterministically), plus unparsed payload and unittest facts when
  d_extract is on. A packet holds the sealed task text, the verbatim clause
  texts, extraction requests, deterministic source excerpts and the scrubbed
  answer; under 50,000 bytes; ids follow the content hash, so controls are
  mixed blind. A memory task carries only the frozen record digest and the
  anti-pattern row.
- Scrubber: one linear left-to-right pass replacing exact run values, UUID
  and tool-call id shapes, absolute paths and the 81 denylist, model, family
  and arm names, keeping spans so an extraction quote maps back to the raw
  answer; the canary (values and id shapes) runs at packet time and again at
  dispatch.
- Checks: every quote must be verbatim in the packet (unknown judge_quote), a
  leak is unknown judge_leak, a passing judgment gets one same-family
  refuter (a stood refutation is unknown judge_refuted), a command, MCP, web
  or file item is unknown judge_audit, at most one retry and only for
  infrastructure failure, a usage limit pauses the run (exit 75, resumable,
  the failed attempt retained). Calibration per template and route: any
  misjudged or unjudged control makes that template's judged clauses unknown
  judge_calibration; extraction controls gate the extractions only.
- Codex route: gpt-6-astra at the literal max effort (JUDGE_EFFORT; the lane
  default is never referenced), ISOLATION_ARGS, a run-scoped home with an
  empty HOME, BLIND_CHILD_PATH, the strict schema, the packet inline and an
  empty per-item -C directory; E_JUDGE_ISOLATION for a home, PATH or launcher
  problem; the live quota is recorded, never a gate.
- Claude route: claude-args (scriptPath, Workflow args, snapshot, the
  coordinator request that reads one small export file and runs one Grep
  first), E_JUDGE_ROLE unless the user-scope blind-lane-reviewer copy is
  byte-identical (checked: both copies exist and match), collect (snapshot,
  prompt, repo and item binding, transcript_audit, zero hook rows).
- Windows: every dispatch refuses while either run window is open
  (E_WINDOW_OPEN window=W_C or W_X); rehearse runs before any window.
- rehearse: two planted controls (one known pass, one known fail) through a
  route; a receipt of counts and booleans, the raw judgments and versions
  private. The real routes are not run here (no network): acceptance e8 open.

evidence.py: id_shape (UUID, tool_use or call id with eight or more characters,
home slug); judges.routes in the aggregate (calls, judgments, refuted, leaks,
calibration_failed_templates, unavailable, usage_recorded, the last recorded
outside G-C); the optional controls block; run_controls, the planted-control
runner for acceptance e5 (class A by oracle, class B for T31, T14, T0 and T27,
class C for the carrier functions).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The remaining grade.py subcommands (design b3, R21 and R22).
- judge {packets, codex, claude-args, collect, rehearse}: thin dispatch to
  judge.py with grade.py's shared refusals (private outputs, spec
  regeneration). `grade --judgments` and `regrade --judgments` take a file or
  a judge directory holding judgments-*.jsonl; route summaries travel in the
  judgments and publish under judges.routes.
- controls: planted class A, B and C controls from calibration/oracle-controls
  .json, the two committed E1 answers graded at their pinned keys (the repomix
  47-name list fails T8, the qmd wrong-document answer fails T4) and the
  calibration files checked against the registry and the grading block's
  minimums; counts only, exit 0 when every class landed where it was planted,
  1 otherwise. `--out` keeps the counts and `grade --controls` publishes
  them under `controls`; regrade repeats them from the private copy.
- differential: this grader's T0 oracle against the retained check.py of
  rtk-296 and rtk-343 on the original baseline and three mutations (subject,
  test count, result), each retained check run from a temporary copy with its
  hard-coded BASE path pointed at the copy; counts only. Run here on the real
  retained inputs: `rtk-296 agree 4/4; rtk-343 agree 4/4`.
- export: one returned_results record (schema_version 1 with the top-level
  captured_at and scope the token-report importer requires) whose attachment
  is the ID-free aggregate, in a fresh neutral directory; the canary runs over
  the aggregate and the whole manifest, and the directory is refused
  (E_EXPORT_DIR) inside a work tree, when it exists, or when its own path names
  a run value, the home directory or the user name. The recorded command is
  the grade or regrade command with a placeholder for every private path
  (context.json now keeps it).
- check-html: the value canary (with --from a grade run's private directory,
  the run's own values; always the home directory and user name), UUIDs,
  tool_use and call ids and the home-slug shape. The '/' shape is not applied:
  URLs and tags carry it. Without --from no run value is known (recheck low
  finding on the canary source).
The fake codex in the tests answers `--version`.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The blind calibration controls must show the judge the same source sections
its real packets carry, so a control that passes or fails says something
about the real route. The controls of T5, T12, T13, T14, T16-T25 and T28 are
regenerated through judge.source_sections (the section labels now read
"... lines naming the facts"), and the T13 packet gains the recorded
conversion facts (call state, converted bytes, scope terms missing from the
conversion output) without the tool name.

F29b gains a guard that the real report templates of
tools/token-report (token_manifest.html.in, token_manifest.full.html.in) and
returned_results.js pass the check-html canary with neutral values. It passes
on first run by design: it pins a property of upstream files, not new code.

Sources: tools/token-e2e/judge.py source_sections, tools/token-report
templates at this base, the U9 design R7 (calibration) and c2.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Two design items stage 2 left unbuilt, each with its tests first.

F41 (R2-06): the decided reading grades the answer as Markdown (the ten
structure checks); the published alternative grades the content of the same
ten elements wherever they sit, read from the fixture at the exec revision.
F42 (M9): the optional block reports, per seeded optional task, the arm-B
attempts and whether each used the task's lane tool (repomix, markitdown).
U4's join lanes are the eight required tool lanes, so the lane is read from
the attempt's own calls and the call ledger's state.

Failing values at the parent (8 tests, 4 failures, 4 errors):
- F41 key test: KeyError: 'contents'
- F41 plain report: ('fail', {'elements_missing'}) != ('pass', set())
- F41 left-out element: missing lists all ten kinds, expected ['image']
- F41 whole word: missing lists all ten kinds, expected ['table']
- F41 key without contents: ('fail', {'elements_missing'}) != ('unknown', {'key_missing'})
- F41 published alternative: KeyError: 'T38' (ORACLE_READINGS has no entry)
- F42 both tests: KeyError: 'use'

Sources: preregistration M9 (report use and correctness, never gating),
scripts/native_token_ci.py markdown_elements (unchanged upstream), and
fixtures/markitdown-multi-element.html.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
R2-06: key_T38 now carries the content of the fixture's ten elements
(html.parser over the exec-revision fixture, document order, script, style
and comment text excluded), and oracle_T38 grades the published alternative
element_contents by reading each content string as a whole word in the
normalized answer. The decided reading (the ten markdown_elements checks) is
unchanged. ORACLE_READINGS gains T38, so the alternative is evaluated and
published, not a copy of the decided result. A fixture without one of the ten
elements leaves the key unknown instead of making its check vacuous.

M9: the optional block of the aggregate gains `use` per seeded optional task
(lane, attempts, adopted, not_adopted, unknown), counted per arm-B identity
from the lane tool the attempt's own calls used (repomix or markitdown as a
whole command word, or an MCP server of that name) with the call ledger's
state deciding success. U4's join lanes are the eight required tool lanes, so
they carry no state for these two lanes. Reported beside correctness, never
gating (sealed M9).

The eight F41 and F42 tests pass (5 s).

Sources: preregistration M9 and R2-06 of the U9 design (R5 register),
scripts/native_token_ci.py markdown_elements (unchanged upstream).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The design's traceability table (c4) promises aggregate fields that no
earlier test named: blind.verdict_parity, team_T.unmapped_teammates,
attempts.superseded_runs, representation_sizes.payload_bytes,
m13_organic and the optional block's use shape. F43 asserts each of them on
a synthetic run: a workflow and strict verdict that agree, disagree and are
absent; a teammate whose transcript is gone at grading time (tasks 1,
unknown 1, unmapped 1, outside the G-Q population); a superseded usage-limit
run beside its final attempt; the payload size of the web answer; the empty
organic binding block; and one binding child without a sentinel record
(unknown 1).

These describe behaviour that was already built, so the six tests are
guards and pass on their first run (7 s). The own and sibling counts of
m13_organic are not driven end to end: the shared bindings fixture carries no
sentinel record for seed-binding-1, and the T36 oracle itself is covered by
F10_Binding.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
tools/token-e2e/README.md gains one section for grade.py (design f6): the
command flow, the four grading classes and the attempt classes, the two blind
judge routes and the reasons a judgment becomes unknown, the planted
controls and the T0 differential, export and check-html, the privacy rules
and exit codes, the limits (real judge routes not run, Amendment 4 pending,
recorded residual on quoted hit text) and the obligation on the report
producer: neutral origin and path values in the data handed to
render_reports, then check-html --from on the result (recheck RV-14). The
section holds no table row that starts with a backticked id, so U8's README
catalogue test is unaffected.

docs/decisions/2026-09-29-frozen-check-grader.md records the choice of
standard-library Python with unittest over promptfoo 0.123.1 (llm-rubric,
g-eval) and Inspect (model_graded_fact, model_graded_qa), names the rejected
scorers with their evidence, the overturn conditions, the evidence classes and
the calibration alignment (review RV-28).

Sources read on 2026-09-29: npm view promptfoo (0.123.1, modified
2026-09-18), pip index inspect-ai (0.3.272), the promptfoo llm-rubric and
g-eval documents, the Inspect scorers document, and the U9 design a5 and f6.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The README limits, the decision record and a test comment said the unit had
no network. That was wrong: the unit made read-only fetches (npm view, pip
index, the promptfoo and Inspect documents). The real reason is scope: its
brief allowed read-only document fetches only, so no judge model call was
made. Acceptance e8 (grade.py judge rehearse on both routes) needs real judge
calls and stays open.

Wording only; no behaviour changes.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
scripts/validate.py refuses a tracked file that contains a personal home path
(the pattern exempts /home/example). The scrubber test used a synthetic
path under another user name to check that home paths become <path> and map
back to the raw text; it now uses /home/example/work/file.py. The scrubber
treats every absolute path alike, so the expectations are unchanged and the
two scrubber tests pass.

Found by running validate.py for the manifest registration, the first time
the U9 files were in scope of that check.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Last commit of the branch, per the hot-file protocol of docs/lanes.md: the
entries come from host_receipts.register_file, so files[] stays sorted and
only the new entries and the README's hash and size change (33 entries added,
tools/token-e2e/README.md updated because U8 registered it; +167 -2 lines).

Registered: the ten tools/token-e2e sources, schemas, template and README,
the 22 calibration files, tests/test_token_e2e_grader.py and, taking design
e7 ("the new files") at face value, the decision record.

python3 scripts/validate.py passes ({"status": "passed"}) and git diff
--check is clean. component_matrix.py --check and new_host_grand_list.py
--check also pass, so neither generated report needed a rewrite.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Two high and six medium findings of the independent review of head 705df138.
Each test below fails at that head for the stated reason; guard tests that
pass at the parent are marked (guard). Command: python3 -B -m unittest -v
tests.test_token_e2e_grader.<class>; observed values, host paths removed.

M7 answer payloads (high 1) and toon commands (medium 5), class F28_M7:
- unfenced TOON answer plus a "latency sum" line: answer items
  ['equal', 'unequal'] != ['equal']; a wrong or broken-header unfenced payload:
  ['unequal', 'unequal'] != ['unequal']; m7 round trip
  (checked, equal, rate_lower, rate_upper) = (2, 1, 0.5, 0.5) != (1, 1, 1.0, 1.0).
- `toon --stats seed.json && toon -d out.toon` and the tee chain:
  (cli_encodes, encode, ineligible) = (0, 'not_encoded', 0) != (1, 'encoded', 0).
  A pipeline ending in `toon -d`: (0, 'not_encoded', None) != (1, 'unknown',
  'unparsed'). `curl -d ...; toon --stats`: (0, 'not_encoded', None); `sort -o
  ...; toon --stats`: (1, 'unknown', 'output_file') != (1, 'encoded', None).

Fields named by their field name (high 2), classes H3_FieldNamedFacts and
H3_PositiveControlOracle (medium 3):
- T32 "id 1, latency_ms 17", "record_id: 1, latency_ms: 17", "id=1 latency_ms=17",
  JSON keys: ('unknown', {'unparsed'}) != ('pass', set()); latency_ms 18:
  unknown != fail(latency). Sums "latency_sum: 124", "Sum of latency_ms: 124",
  "total_latency_ms = 124", "The total latency is 124 ms", "Sum of latency: 124":
  ('unknown', {'unparsed'}) != ('pass', set()); "latency_sum: 125" is unknown,
  not fail(latency_sum). spaced_identifiers does not exist.
- T33 "all INFO (no ERROR among them)" and five more negations:
  ('fail', {'levels'}) != ('pass', set()); quoted raw rows in four shapes:
  ('fail', {'events'}) != ('pass', set()); raw rows with one ERROR level:
  {'events', 'levels'} != {'levels'}; levels list before events list: unknown.

D-extract contract (medium 4), F7_T0 and F28_M7:
- number words through the production check_judgment:
  ('unknown', {'unparsed'}) != ('pass', set()); a stub whose values are not
  inside its quote: ('pass', set()) != ('unknown', {'judge_quote'}), for the
  unittest and the payload component; spelled_numbers_to_digits is missing
  (9 subtests error). The stub of the old F7 test and the header-only quote of
  the old payload test are replaced by extractions the production contract
  accepts (guard).

Historical memory (medium 4), F25b_MemoryHits: a bare slug, a later page that
names recipes/<slug>.md, and three longer paths that contain the frozen path:
('pass', set()) != ('fail', {'no_historical_hit'}). The old test that expected
a bare slug to resolve is inverted.

Canary scope (medium 8), F17b_CanaryScope, F29b_CheckHtml, F35_JudgeRoutes:
'workflow_child' and the anchor line are in the canary list;
ev.IDENTIFIER_FIELDS is missing; check-html and `judge packets` exit
2 != 0 with E_PRIVACY for an actor name and for a quoted instruction line.

Argv sanitizer (medium 6), F29c_ArgvSanitizer: `--flag=value` is recorded
verbatim for all 17 path flags ('--spec=<path>'), a full grade run writes the
path into context.json, an abbreviated `--out-priv` is accepted ('' !=
'E_ARGS field=usage', exit 1) and export copies a recorded '--spec=/x' into
the manifest ('' != 'E_EXPORT_INPUT reason=argv', exit 0).

Guards that pass at the parent: an unfenced payload alone is one equal trip,
a fenced broken header is one unequal trip, the toon command's own -o stays
unknown, a heredoc decode after `cut -d`, the underscore stays a word
character, a count field is not a sum, hedged and wrong values still fail.

Sources: the review and recheck findings of the U9 stage-3 head; TOON CLI
4.1.1 strict decode (content after a root array is refused); check_judgment in
judge.py (the production contract of an extraction).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
High finding: _answer_toon_items counted the failed strict decode of the
whole-answer candidate as an unequal round trip. A correct unfenced TOON answer
with the requested latency sum after it (TOON CLI 4.1.1 refuses content after a
root array in strict mode) therefore had one equal and one unequal item, and M7
failed on both bounds. Now only a structural candidate (a fence or a line
block) that starts like a TOON header and does not decode is a failed trip; a
bare candidate (the whole answer, an evidence string) that does not decode adds
no item, as frozen_checks.grade_payload already ignores it. Without a frozen
original the same split applies (one unknown per payload, not one per candidate).

Medium finding: toon_facts read `-d`, `--decode`, `-o` and `--output` from every
word of a call, so `toon --stats seed.json && toon -d out.toon` counted as a
decode only (no encode, no round trip) and `curl -d` or `sort -o` beside an
encode changed its meaning. toon_invocations now reads each simple command of
the call separately (the existing _shell_commands and _program_of scanners):
a call may hold an encode and a decode; an option belongs to the command it
follows; a heredoc feeds a round-trip item only when the decode command itself
carries it. A script inside `bash -c '...'` still falls back to the words of
the whole text.

Sources: R14 of the U9 design (a seeded encode is a succeeded toon call whose
result holds a document that strictly decodes to the seeded array; an
unresolvable result is unknown), correction 5 and 6 of the build corrections,
toon CLI 4.1.1 (strict decode, assertFullyConsumed). Tests: F28_M7 (9 new
cases and the whole class: 40 tests, all pass except the payload-extraction
case that belongs to the D-extract commit).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
High finding: word_positions keeps '_' a word character (T1's whole-word
reading, grep -w semantics, depends on it), so a label never matched inside a
field name. A correct blind answer "Verdict: yes. id 1, latency_ms 17" gave
latencies [] and unknown(unparsed), and a correct web-table answer
"latency_sum: 124" or "Sum of latency_ms: 124" left the sum unparsed after the
payload had decoded. The D-extract fallback cannot decide either case (it
covers T0 and the payload only), so an arm-B unknown failed G-Q clause 1 on the
lower bound.

The word rule is unchanged. The label sets now name the facts the way the
frozen key does:
- T32 reads `id` and `record_id`, and `latency`, `ms` and `latency_ms`; a label
  that is a field name (it holds an underscore) takes its value after it only,
  so `id=1 latency_ms=17` never reads the 1 as a latency.
- ints_by_label skips the closing quote of a quoted field name, so a JSON
  answer `{"id": 1, "latency_ms": 17}` is read.
- The stated sum reads its line through spaced_identifiers (an underscore
  between two letters reads as a space: latency_sum, total_latency_ms) and lets
  the sum or total label be followed by the latency nouns and the linking words
  ("Sum of latency_ms: 124", "The total latency is 124 ms"). A count field
  (total_records: 8) is still no sum; a hedge ("124 or 125") stays unparsed and
  a wrong value still fails.

Sources: design R3 (recognised forms; an unparsed key fact is unknown, a
contradicting value is a failure), R4 (the requested latency sum), the frozen
key fields record_id and latency_ms of fixtures/table.json. Tests: the new
H3_FieldNamedFacts (7 cases) and the oracle classes G_TemplateSweep,
H_OracleReviewFindings, F6_StrictDecode, F32_TokenLines, F5, F9, F10 and F18
(81 tests, all pass).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Medium finding: oracle_T33 failed correct answers to seed-blind-positive, which
G-Q clause 1 counts, and a FAIL counts on both bounds (R3: FAIL means
demonstrably wrong). "Events 1, 2, 3, 4, 5, all INFO (no ERROR among them)" gave
fail(levels) because any whole-word ERROR failed, and an answer that quoted the
raw rows `[{"event":1,"level":"INFO","value":1,...}, ...]` gave fail(events)
because every integer of the first bracket span was taken and the fixture rows
carry value == event.

- level_is_negated: a level word is the object of a negation when walking back
  from it over separators (spaces, commas, slashes), filler words and other
  level words reaches a negator (no, not, none, nothing, without, zero, never,
  neither, nor, 0, n't). Any other word or punctuation stops the walk, so
  "the third is not INFO, it is ERROR" and "none of them is WARN, but the fourth
  is ERROR" are still levels that are there.
- The events are read from quoted raw rows (`"event": N`, either quote), else
  from the first bracketed list that holds only integers, else from the lines
  that name events. A levels list before the events list no longer hides the
  events; a wrong event or level in raw rows still fails.
Both scanners are linear character scanners.

Sources: design a4 row T33 ("Events 1-5, all INFO"), R3; the sealed check text
of seed-blind-positive and the fixtures/events.jsonl row shape at c0966da.
Tests: H3_PositiveControlOracle (5 cases) and the oracle classes; F36_Controls
(planted controls) still lands every control at its expected status.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Medium finding: the F7 D-extract pass rested on a stub judgment that the
production contract rejects. Its values ['49 tests', 'OK', 'skipped=1'] are not
substrings of its quote 'forty-nine tests, OK, 1 skipped', which
judge.check_judgment answers with judge_quote. With verbatim values the grader
read the same words with the same grammar, and count_before_noun leaves
hyphenated number words unparsed, so in production the T0 fallback could never
change an outcome. Neither _apply_extraction nor _extracted_payloads checked
that a value lies inside a quote, although check_judgment's docstring says the
oracle verifies them again; a `grade --judgments` or `regrade --judgments` file
could therefore inject a fact the answer never states.

- verified_extraction: a D-extract item is usable only when every quote is
  verbatim in the raw answer and every value is a substring of one of its quotes;
  otherwise the component is unknown(judge_quote). Used by the unittest and the
  payload fallback.
- spelled_numbers_to_digits: the extraction-only reader of number words up to 999
  (forty-nine, twenty one, one hundred and five). Grammar g1 is unchanged, so an
  answer stated in words is still unknown(unparsed) until a judge quotes it
  verbatim, which is the design's F7 case; the quote is then read with this reader.

The old F7 stub and the header-only quote of the payload test are replaced by
extractions the production contract accepts (F7 now builds its extraction with
judge.check_judgment); the injected variants of both are new tests.

Sources: design R3 and R21 (an unparsed key fact goes to D-extract; the oracle
verifies quotes again), F7 of the design, judge.check_judgment. Tests: F7_T0,
F7b_T0Rerun, F28_M7, F15_JudgeContract and F23_WebTableJudges (81 tests, all pass).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Medium finding: resolves_to_record also matched a frozen record by its bare
path stem, so a later page that only mentions the same slug counted as a
historical hit. The retained ai-memory output shape uses title-slug paths
(`decisions/<slug>.md`, title `<slug>`); with a frozen
`decisions/host-request-lane.md`, a later session page whose snippet reads
`... recipes/host-request-lane.md ...` matched on `/host-request-lane.` and
memory_check returned ok. R9 and U9-D13 count a hit only when the result names a
frozen record, precisely so that a later arm's page never counts (arms B, A and
A0 share one store). The F25b fixture used the id-like stem `mem-fx-1`, which
hid it.

resolves_to_record now accepts the record's content digest or its whole path.
The path is a unit: nothing that belongs to a name (a word character, '-', '.'
or '/') may sit before it or after it, so `archive/<path>`, `old-<path>` and
`<path>-old` are other pages. The bare stem no longer resolves. The freeze
records a path and a digest per record because `ai-memory search --json` rows
carry no id (retained receipt: rows hold path, title and snippet; memory_query
hits hold id and path), so no id field is added here.

The F25b test that expected a bare slug to resolve is inverted; a later page
naming another file with the same base name, and three longer paths that contain
the frozen path, fail with no_historical_hit, and the whole path and the digest
still pass.

Sources: design R9 and U9-D13; the retained receipt
evidence/artifacts/gap-wave2-20260923/recovery-portability (ai-memory search and
memory_query output shapes). Tests: F25b_MemoryHits, F25a_MemoryKeys,
F4b_RecoveryEvidence (23 tests, all pass).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Medium finding: sanitize_argv replaced a path only when the token was exactly a
PATH_FLAGS key followed by a separate value. argparse also accepts
`--spec=/tmp/run/spec.json` and any unambiguous prefix of a long option
(`--out-priv /p`), so a `grade --spec=/some/private/spec.json ...` run stored
the path in context.json and `export` copied it into the argv of
returned-results.json. The export canary missed it: has_private_shape flags
only a string that starts with '/' or contains ' /', and a private path outside
the home directory is not a gathered value. R22 and U9-D17 require the exported
argv to hold only placeholders.

- sanitize_argv reads both spellings (`--flag value` and `--flag=value`, with the
  `FAMILY=FILE` value keeping its family: `--join-ledger=claude=<join-ledger>`).
- Parser sets allow_abbrev off for the parser and every sub-parser (they inherit
  the class), so an abbreviated flag is E_ARGS field=usage before anything is
  written and the recorded flag table is complete by construction.
- export refuses (E_EXPORT_INPUT reason=argv, nothing created) a recorded
  command that is not a list of strings or whose elements after the script path
  hold a '/', a backslash, a home shorthand or a drive prefix: the fail-closed
  backstop for a context.json written by any other means.
evidence.has_drive_prefix is the former private `_has_drive_prefix`, now shared.

Sources: design R22 and U9-D17 (export argv holds only placeholders), argparse
(allow_abbrev: "allows long options to be abbreviated if the abbreviation is
unambiguous"). Tests: F29c_ArgvSanitizer (5 cases: both spellings for all 17
path flags, a full grade and export with `=` spellings, an abbreviation, the
backstop); with F29_Export and F29b_CheckHtml 18 tests run and 17 pass, the
remaining one being the canary-scope case of the next commit.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Medium finding: canary_values gathered strings that are not identifiers, so
legitimate outputs were refused with E_PRIVACY. It added every string field of
every identity row, which includes the actor vocabulary of 8 or more characters
(workflow_child, strict_process, agent_child, codex_exec, codex_subagent,
team_teammate), and the instruction-file anchor lines. R22's canary covers only
the run token, identities, labels, ids, events file names, input paths and
roots, the home directory and the user name. So `check-html --from <private
dir>` refused a report that printed an actor name, and in `judge packets` one
answer that quoted a line of AGENTS.md or CLAUDE.md refused the whole batch (no
packets written, every D clause pending).

- evidence.IDENTIFIER_FIELDS is the one closed list of identifier fields of an
  identity row (identity, the locators, ids and the label); canary_values reads
  only those (a path-like locator also by its last name, now including
  parent_events_file), plus the run token, the records' identity and agent id,
  the roots, the memory roots, the exec checkout, the arm and Codex paths, the
  sentinel values, the home directory and the user name.
- judge.ID_KEYS is that same tuple, so a value the packet scrubber removes is a
  value the canary checks; the canary stays the second guard behind the scrubber.
- The instruction anchors stay in the bindings for M12 test 3 and are no longer
  gathered.

Sources: design R22 (aggregate canary values). Tests: F17b_CanaryScope (3 cases),
F29b_CheckHtml (actor vocabulary passes, a planted run token still refuses),
F35_JudgeRoutes (a quoted instruction line still yields packets), F17, F29 and
F35 (37 tests, all pass; the canary-off mutants M17, M17c, M48 and M50 are
untouched).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
U9-D19: every guard gets a condition-absent run. Each mutant patches one repair
guard back to the behaviour the review found and asserts the exact set of
failing tests; the guard tests beside it must stay green, so a mutant is
minimal (11 mutants, all pass on their own expectation):
- M52 a bare candidate that does not decode counts as an unequal trip
  (the stage-3 head's _answer_toon_items): 3 tests flip.
- M53 toon options read from every word of the call (toon_invocations off):
  3 tests flip; the toon command's own -o and a heredoc decode stay green.
- M54 an extraction value outside its quote is accepted (quotes-only
  verified_extraction): the unittest and the payload injection tests flip.
- M55 a frozen record matched by its bare stem (the head's resolves_to_record):
  the bare slug, the later page and the three longer paths flip.
- M56 `--flag=value` recorded verbatim (the head's sanitize_argv): 3 flip;
  M56b abbreviations accepted (allow_abbrev on): 1 flips;
  M56c export backstop off (argv_issue): 1 flips.
- M57 the canary gathers every row string and the anchors: the vocabulary test,
  check-html and `judge packets` flip.
- M58 field names not read through their words (spaced_identifiers off),
  M59 a negated level word is a level, M60 number words left as words.

Sources: design b5 F19 (one disarmed-guard mutant per guard) and U9-D19; the
stage-1 to stage-3 mutant classes F19, F19b and F19c (same failing_with
harness).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The README section of tools/token-e2e/README.md (the grader's own section only)
now states what the repair round changed for a reader of the tool:
- how a fact is read (labels, field names, a quoted JSON key, negated level
  words, quoted raw rows), and exactly where the D-extract fallback applies:
  the T0 test-run words (read with an extraction-only number-word reader) and the
  payload of T9 and the web-table tasks. The web-table sum and the blind tasks'
  facts have no fallback and stay unknown(unparsed); a quote is used only when
  it is verbatim in the answer and every value lies inside a quote, whichever
  path the judgment came from.
- the canary scope (identifiers and paths, not actor vocabulary or instruction
  lines), the two flag spellings, refused abbreviations and the export backstop.
- two limits: a historical memory hit is a whole path or a digest (real result
  shapes for the frozen records are unmeasured), and a payload extraction may be
  requested for an answer whose payload decoded and only whose sum is unparsed,
  which the grader ignores.

The decision record gets a "Repair round" entry: the eight findings, the
choices, and the withdrawal of the earlier claim that the D-extract fallback may
decide an unparsed sum or blind-task fact (it covers neither). Extending it to
them needs new extraction controls and is left to the Amendment 4 review.

Sources: the review of head 705df138 and the design's R3 and R21.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Found by the independent check of the repair round before the manifest commit.

R2-16 regression of 6f280d97 (F28_M7): with no frozen original, an undecodable
structural block now added an unknown round-trip item whatever R2-16 says. The
stage-3 head added such an item only under `unknown_in_denominator`, so the
published "excluded" alternative counted payloads it must exclude. The existing
R2-16 test used a document that decodes, which hid it. New test, one payload
with no original in three shapes (unfenced broken block plus a sum line, the
same block valid, a fenced broken block): decided reading ['unknown'], excluded
reading []. Observed at this head: ['unknown'] != [] for the unfenced and the
fenced broken block (the valid block passes). At 705df138 the unfenced case
gave two unknown items for one payload.

Scanner cost of the T33 repair commit (H3_PositiveControlOracle), whose message
calls both scanners linear:
- integer_lists searched again for `]` from every `[`: 6001 str.find calls on
  `[` * 3000 + "7, 8]" (bound 50).
- level_is_negated walked back over every earlier level word (quadratic on
  `no ERROR ERROR ...`): with 9 words the walk still reaches the negator,
  observed True != False (a walk over more than eight words is no longer one
  clause).

Sources: design R14 and R2-16 of the U9 register; the repair-round check.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
R2-16 regression of 6f280d97: _answer_toon_items decoded every candidate first,
so with no frozen original an undecodable structural block added an unknown item
even under the "excluded" reading. The records-None case is decided before any
decode again, by the reading alone: unknown_in_denominator adds one unknown item
per payload, anything else adds none. A bare candidate (the whole answer, an
evidence string) counts there only when it decodes, so an unfenced payload
followed by a sum line is one payload, not two (at 705df138 it gave two unknown
items). A payload with an original is unchanged: a structural block that does
not decode is one unequal trip, a bare one is no trip.

Scanner cost of the T33 repair commit, whose message called both scanners linear:
- integer_lists is one pass now: each `]` closes the innermost `[` still open,
  so a run of open brackets no longer searches again from each one (6001 find
  calls on 3000 open brackets before; the pairing is the same).
- level_is_negated stops after eight words (_NEGATION_WALK): a negator further
  back is no longer in the same clause, and a walk no longer costs time in every
  earlier level word (quadratic on `no ERROR ERROR ...`). Eight words reach every
  negation of the tests, including `nothing at WARN or ERROR level`.
The remaining quadratic scanners named in the review, code_spans and _or_links on
very long lines, are unchanged and stay listed as open.

Sources: design R14 and the R2-16 row of the U9 register. Tests: F28_M7,
H3_PositiveControlOracle, H3_FieldNamedFacts and the mutant classes F19b and
F19d (81 tests, all pass).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
U9-D19 for the guards of the previous commit, in F19d_RepairRoundMutants:
- M52b decodes before asking R2-16 (the regression itself): the
  no-original round-trip test flips and the two unfenced-payload tests stay green.
- M61 the quadratic integer_lists of the T33 repair commit: the one-pass test flips; M61b an
  unbounded negation walk (_NEGATION_WALK patched to 10**9): the clause-bound test
  flips. The held test in each is the list-reading test.
13 mutants in the class, all pass on their own expectation.

Sources: design b5 F19 and U9-D19.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Last commit of the repair round, per the hot-file protocol of docs/lanes.md: the
seven registered files that changed since 705df138 get their new hash and size
through host_receipts.register_file (files[] stays sorted; 14 lines change, no
entry is added or removed): frozen_checks.py, evidence.py, grade.py, judge.py,
tools/token-e2e/README.md, tests/test_token_e2e_grader.py and the decision
record. The calibration files, both schemas, the judge template, the Workflow and
node_bridge.mjs are unchanged, so their entries stay.

python3 scripts/validate.py passes ({"status": "passed"}), git diff --check is
clean, and component_matrix.py --check and new_host_grand_list.py --check pass.

Integration: this branch now holds two manifest commits (705df138 and this one).
At rebase take main's manifests/evidence.json and re-register the U9 files again
(scripts/host_receipts.register_file), as the protocol says; never hand-merge it.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 29, 2026
@seathatflowsinourveins
seathatflowsinourveins merged commit a02ff13 into main Sep 29, 2026
25 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/pra-u9-frozen-checks-2d-20260929 branch September 29, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant