Skip to content

macOS token-efficiency pins for eight tools, uv-tool installers, rtk macOS config path (GPT-6 reviewed) - #339

Merged
seathatflowsinourveins merged 4 commits into
mainfrom
claude/macos-token-pins-20260926
Sep 26, 2026
Merged

seathatflowsinourveins merged 4 commits into
mainfrom
claude/macos-token-pins-20260926

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Pins: the macOS pins gain rtk, qmd, repomix, toon, ccusage, headroom, markitdown and serena at their Linux versions. macOS pin coverage goes from 6 of 14 to 14 of 14 for token-efficiency, and from 5 of 7 to 7 of 7 for foundation-cpu.
  • Script: bootstrap-macos.sh gains the Linux uv-tool and uv-tool-from-git install functions, byte-identical and tested under bash 3.2. It also prints an rtk reminder that checks ~/Library/Application Support/rtk/config.toml, the file rtk 0.50.0 reads on macOS.
  • Evidence (all local_integration; no Mac ran anything):
    • digest re-check: 61 checks, 0 mismatches;
    • negative control: 16 mismatches, then serena's commit refused;
    • rtk config-path source read and pinned-binary read-back: PASS: all 16 checks held;
    • pins-file field comparison since the digest run;
    • red/green: 14 failures before the fix, 13 in the mutant, all 12 passing when fixed;
    • fail-closed controls: 3 of 3 hold against the committed helpers, 0 of 3 against the earlier ones.

Follow-up (2026-09-26, GPT-6 + independent Claude verification): Ported bootstrap-linux.sh #334's wheel-download-and-verify install_uv_tool to macOS. Extracted a shared verify_sha256() helper (prefers shasum -a 256, GNU sha256sum fallback) so fetch(), verify_sha256(), install_uv_tool() and install_uv_tool_from_git() are now byte-identical between the two bootstraps; headroom's macOS wheel is downloaded and sha256-verified before uv runs, matching Linux. Docs and evidence updated with changed after v2026.09.26`` markers. Test coverage extended (PORTED_FUNCTIONS now covers 4 functions; new real-bash-3.2 tests for both checksum-tool branches and the full wheel-install path). Verified: 267/267 named tests, bash -n + shellcheck clean on both scripts, `validate.py`/`evidence_manifest.py --check`/`validate_catalogs.py` all pass, full suite 6172/6173 (the one failure is a pre-existing host-state artifact unrelated to this change — see verification notes, not a code regression).

Review

  1. Build: Sonnet, followed by a program reviewer.
  2. Opus port and repair: switched to rtk 0.50.0 hook recipe: exclude git show REV:path in every global-option form and git branch #318's four-entry rtk block.
  3. Independent Opus review: 1 high (rtk reads ~/Library/Application Support/rtk/config.toml on macOS) plus lows, fixed in polish.
  4. GPT-6 cross-family review: 2 medium, both verification helpers failing open. Fixed so they fail closed, and re-run.

Residuals

  • Resolve manifests/evidence.json by taking main's copy, re-registering this change's files per docs/lanes.md, then running component_matrix.py --write and new_host_grand_list.py --write.
  • Resolve step 4a by keeping main's line (with Fix Codex post-merge findings: wider rtk blob exclusion, TOML-parsed bootstrap reminder, blind-packet leak, retained markitdown evidence #314's rtk hook check sentence) and replacing "in ~/.config/rtk/config.toml with" by this change's "in rtk's config file (…) with" parenthetical.
  • Validators on a resolved tree were last run at b5c6313b, not at 2e771442.
  • The macOS reminder checks only the config text. It does not run Fix Codex post-merge findings: wider rtk blob exclusion, TOML-parsed bootstrap reminder, blind-packet leak, retained markitdown evidence #314's rtk hook check probe; the macOS page tells users to confirm with rtk config and the recipe's rtk hook check commands.
  • The macOS config path comes from rtk 0.50.0's source and docs, not from a Mac. RTK_CONFIG_PATH_REVIEWED_VERSIONS fails the suite on an rtk re-pin until the source is re-read.
  • No Mac ran anything. macos-arm64 stays drafted_not_accepted. The digest checks, config-path check, red/green run and fail-closed controls are local_integration harnesses written for this change. The controls are synthetic wrong inputs, not upstream tests. No host receipt was recorded.
  • verify_pins.py was not re-run. The pins files have not changed since pin-fields-since-run4.txt.
    • It still has a variant of finding 2: when packaging is not importable it logs and skips headroom's requires_python check instead of failing.
    • Every retained run that reached headroom shows that check line, so no retained result depends on the skip.
    • Left unfixed because the fix needs a fresh networked digest run.
  • Coverage of the new checks. The controls exercise only the crate-digest refusal, a non-3.2 bash and skipped tests. The rtk check's tag, fetched-commit, lock-entry, release-digest, binary-result and listing checks are enforced but have no wrong-input control; scratch sanity checks of the lock parser, expect and the verdict function were not retained.
  • Fail-open helpers kept. rtk_config_path_check-run1.sh and reminder_path_red_green-run1.py stay byte-identical: they produced the run-1 outputs and are the "before" side of the controls.
  • Unpinned and unchecked. qmd's sqlite-vec-darwin-arm64 and ccusage's @ccusage/ccusage-darwin-arm64 optional dependencies are unpinned, and uv does not verify the headroom or markitdown sha256.
  • The handbook documents these gaps but does not close them:
    • codebase-memory has no macOS install;
    • socraticode needs an external Qdrant and an embedding endpoint;
    • neither bootstrap starts the ai-memory service;
    • on macOS, adoption_status.py stays prerequisites_missing (exit 2).
  • The test PATH fix was not shown failing first, because this host has no uv in /usr/bin or /bin.
  • Runs not kept (listed in the evidence README):
    • the earlier config-path and red/green runs;
    • eight runs made for this fix between 10:54Z and 10:59Z, each superseded by a retained run;
    • one full-suite run, stopped with no failures shown so it could include a docstring edit.
    • The port report's "3 mutants were each caught" has no retained output.
  • Out of scope, observed. The Linux reminder accepts a relative XDG_CONFIG_HOME, which rtk ignores.
  • Review coverage. The GPT-6 cross-family review covered the change through the polish pass, and its two findings are fixed here. No independent review has covered this repair round.

Verification

  • scripts/validate.py, host_receipts.py validate, component_matrix.py --check, new_host_grand_list.py --check, evidence_manifest.py --check and validate_catalogs.py: all pass on the rebased tree.
  • gitleaks with the repository config over the changed files: clean.
  • Full python -m unittest: see the CI run.

SOTA sources

🤖 Generated with Claude Code

@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 26, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Scout and others added 3 commits September 26, 2026 10:30
…ed with verified digests

rtk, qmd, repomix, toon, ccusage, headroom, markitdown and serena pinned for darwin-arm64 at
their Linux versions; rtk guidance uses the path rtk reads on macOS. Reviewed by Opus and GPT-6.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ls closed without packaging

GPT-6 verification findings on #339: replacing the exclude_commands line with a full block duplicated
[hooks] (invalid TOML, rtk silently used defaults); verify_pins could PASS without the Python
compatibility check. Both fixed with a failing-first test and retained controls.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…verify_sha256 helper

fetch, verify_sha256, install_uv_tool and install_uv_tool_from_git are byte-identical between the
Linux and macOS bootstraps (shasum -a 256 preferred, sha256sum fallback); headroom's macOS wheel is
downloaded and verified before uv runs. Real bash 3.2 tests cover both checksum branches. Fix by
GPT-6 (gpt-6-astra, max), independently verified.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins
seathatflowsinourveins force-pushed the claude/macos-token-pins-20260926 branch from 8ac4251 to 2da621d Compare September 26, 2026 14:32
…asting a second one

Both printed reminders now say to replace the exclude_commands value inside the existing [hooks]
table and add the header only when absent (a duplicate table is invalid TOML and rtk silently
falls back to defaults). Tests fail on the old text. GPT-6 verification finding on #339.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins
seathatflowsinourveins merged commit 5008c30 into main Sep 26, 2026
24 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/macos-token-pins-20260926 branch September 26, 2026 15:07
seathatflowsinourveins added a commit that referenced this pull request Sep 27, 2026
…ta from the 2026-09-27 verdict wave (#401)

* E1/E2 token E2E receipts: dated adjudications, errata and reproduced answer oracles

Applies the 2026-09-27 cross-family GPT-6 review of the E1 (#296) and E2 (#316)
receipts, append-only: every recorded field keeps its value, and each receipt keeps
its own serialization.

- Every tools[] row gains adjudication {as_of, task_acceptance, bad_input_control_recorded,
  basis, evidence}, using the review's status table and vocabulary (partial,
  retracted_later, untested, unsupported).
- E1: QMD retracted_later (wrong_document, corrections_to_296); Repomix retracted_later
  (count=47 PASS on a 48-function file; repomix v1.18.1 PythonParseStrategy.ts L81-86
  tests only a def's first line); ai-memory untested (base_empty=True is a vacuous pass);
  the other thirteen partial with no recorded failing control. A dated errata block
  names the pre-errata sha256 the preregistration records as a historical identity.
- E2: context-mode, jcodemunch, qmd and ast-grep stay partial with no recorded failing
  control; SocratiCode's comparison counts an abridged transcription; RTK retracted by
  its verifier; Repomix's FAIL accurate.
- READMEs carry dated corrections; the review is retained, sanitized, beside E1.
- tests/test_token_e2e_receipt_checks.py: receipt contracts (red before the errata) and
  answer oracles rebuilt from pinned Git blobs, hash-matched to the recorded baselines,
  each with failing controls (red under a weaker historical-style check).

Sources: docs/acceptance-evidence-policy.md (Discriminating controls);
https://github.com/yamadashy/repomix/blob/v1.18.1/src/core/treeSitter/parseStrategies/PythonParseStrategy.ts#L81
evidence/artifacts/token-e2e-codex-20260926/receipt.json#/corrections_to_296.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Token layer: routing rules and recipe limits for known upstream defects

Closes the documentation, recipe and routing gaps from the 2026-09-27 per-tool
verdict wave for the token-efficiency layer. Every rule cites the pinned upstream
source or a committed receipt; no pin changes.

recipes/README.md (token-tool rows and sections):
- agentsview: v0.44.0 unqualified; worker populations need --include-children,
  --include-automated and --include-one-shot, and --fts reads message bodies only
  (v0.43.0 docs/session-api.md, docs/commands.md); archive answers are observation.
- ast-grep: every 0.45.3 command registers customLanguages from any sgconfig.yml in
  the working directory or a parent (lib.rs L107-139, config.rs L98-131/L280-299);
  PR #2960's opt-in is unreleased; outline (#2957) and YAML rule (#2963) limits.
- ccusage: token-only while any model is unpriced (v20.0.24 json-output.md
  Unpriced Models; config-files.md pricingOverrides).
- codebase-memory: trace_path excludes tests and evidence by default, search_code
  hits can be mentions (v0.11.0 src/mcp/mcp.c schemas); caller-set procedure.
- context-hub: check metadata.versions (FastAPI guide 0.136.3 vs 0.141.1).
- context-mode: npm tarball digest and plugin commits are separate identities.
- headroom: v0.39.1 changes only the proxy limiter, so the omission-count defect stands.
- markitdown: HTML as .txt passes through; -x html / -m text/html; selective extras.
- mcporter: --name does not select the server; a positional token becomes the
  selector (v0.14.1 call-arguments.ts L164-192, ephemeral-flags.ts L97-105); the
  jCodeMunch example gains --server.
- qmd: MCP query expands and reranks by default (v2.8.3 README, server.ts); typed
  lex + rerank:false + bounded get; catalog scope; refresh guard for #989/#991
  (src/store.ts reindexCollection).
- repomix: multi-line Python signatures are dropped (v1.18.1 PythonParseStrategy.ts
  L81-86); exact-definition tasks go to an uncompressed pack or the original.
- toon: BOM root-string round trip (#339); no record-count threshold upstream;
  nested-uniform columns are tabular (tabular.ts L61-78).

docs/token-session-handbook.md: a "Known upstream limits behind the lanes" section
outside the carrier text (every carrier line unchanged), and row pointers.
docs/token-practice.md: counts, comparisons and acceptance rules (invocation counts
are not success rates; comparisons need task acceptance; count recovery reads;
gpt-tokenizer's special-token contract; receipts carry adjudications).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Apply the GPT-6 review of this branch: exact inventory oracle, TOON eligibility wording

One cross-family review round (GPT-6 through codex-omniroute, read-only) returned
DEFECTS FOUND with two findings; both are fixed:

- P2 tests/test_token_e2e_receipt_checks.py: inventory_oracle checked only the claimed
  count plus subset membership, so a correct count with missing, duplicated or no names
  passed. It now requires the exact name set with one entry per function. New controls keep
  the correct count while dropping, duplicating or inventing a name, or listing none; they
  failed against the old oracle first (red) and pass now.
- P3 docs/token-session-handbook.md (and the matching recipes/README.md toon row): "accepts
  any non-empty uniform array" overstated TOON 4.1.1's tabular eligibility. The encoder
  needs non-empty objects sharing one key set, with columns of primitives or, recursively,
  of non-empty objects sharing one key set; an empty object or an array-valued column falls
  back to list form (packages/toon/src/encode/tabular.ts L6-78 at v4.1.1; confirmed with the
  installed 4.1.1 CLI on [{}], [{"a":[]}], [{"a":{}}] and [{"a":{"b":1}}]).

Also: the ast-grep oracle keys counts by path under scripts/ rather than file stem, and E1's
Context Mode adjudication states that attempt 1's marker-grep FAIL on a refused call is not
shown to be a run of attempt 2's answer check (the receipt does not record that command).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Apply the independent verifier's findings: TOON source line, citation anchors, E2 Serena erratum

The independent verifier of this branch reported five defects; all are fixed:

- docs/token-session-handbook.md, the TOON Sources line under the carrier (on main
  before this branch) still said "The encoder accepts a non-empty uniform array",
  the wording GPT-6 finding P3 refuted. It now states the encoder's rule from
  packages/toon/src/encode/tabular.ts L6-78 at v4.1.1 and links the handbook's
  "Known upstream limits behind the lanes" section. It is not a "- " carrier line,
  so tests/test_token_lanes_subagent_start.py's verbatim-line check is unaffected.
- Citation anchors: SocratiCode's INCLUDE_DOT_FILES default is documented under
  README.md "Indexing Behaviour" (v1.14.0 L1574-1579), not "Ignore Rules", so the
  handbook links #indexing-behaviour. codebase-memory-mcp v0.11.0 search_code gets
  its own link to src/mcp/mcp.c#L632-L659 in the handbook, and the recipe gives
  index_repository (L466-481), trace_path (L532-560) and search_code (L632-659)
  one link each. Both upstream files matched the retained copies by sha256 on
  2026-09-27.
- E2 receipt errata.items[4] and the E2 README erratum now name Serena's accurate
  reference FAIL (find_referencing_symbols missed 6 of 8 call sites; the definition
  check passed) and give each group of partial rows its own reason. Only that one
  finding string changed in receipt.json (same serialization, 1 line); written_at
  is unchanged because this refines the unmerged 2026-09-27 erratum on the same day.
- tests/test_token_e2e_receipt_checks.py: the require_commit docstring now says the
  guard is modelled on RetainedEvidenceTests' Git check and adds the pinned-commit
  probe (tests/test_adoption_status.py L1721-1724 checks only git and the checkout).

Sources: toon-format/toon v4.1.1 packages/toon/src/encode/tabular.ts;
giancarloerra/SocratiCode v1.14.0 README.md; DeusData/codebase-memory-mcp v0.11.0
src/mcp/mcp.c; the E2 receipt's own serena row (tools[5]) and README.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant