Skip to content

Haiku 5.5 takes cheap exact extraction: record the 2026-10-07 repeat trial (land after 2026-10-08T17:18Z) - #840

Open
seathatflowsinourveins wants to merge 2 commits into
mainfrom
claude/haiku55-extraction-routing-20261007
Open

seathatflowsinourveins wants to merge 2 commits into
mainfrom
claude/haiku55-extraction-routing-20261007

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator

Scope

  • What this PR changes, in one or two sentences: records the 2026-10-07 Claude Haiku 5.5 repeat trial as a sanitized receipt and, because it meets the overturn condition this repository recorded for "Cheap exact extraction | Haiku | Not routed", routes cheap exact extraction (inventories and line-cited extraction that relay no command output) to Haiku 5.5 through a per-call model: 'haiku' on source-scout at effort max. Command running and verbatim output relay stay on source-scout's Sonnet default and judgment stays on Opus. The background-haiku currency row moves to claude-haiku-5-5, and child-usage.mjs gains a haiku alias row.
  • Land no earlier than 2026-10-08T17:18Z (1:18 PM EDT on October 8). That is 24 hours after Claude Code 2.1.293 reached npm (2026-10-07T17:18:04.464Z), the release that made Haiku 5.5 the default Haiku. The command center lands it on cue; it is open for review, not for an earlier merge.
  • Base commit: 36654a8174b5cad1380bd13ed1fffc0f135aac2c (origin/main when the branch was cut).
  • Lane: lane:foundation. This PR also changes what manifests/stack.json says (the Haiku model row moves from claude-haiku-4-5-20251001 to claude-haiku-5-5) and adds a receipts[] entry to manifests/evidence.json. Enforce dated model currency and propose daily source reviews #826 changed the same rows under lane:foundation, so the label follows it; relabel to lane:shared if docs/lanes.md's "changes what a shared hot file says" is read to cover this.
  • Owned paths touched: examples/claude-native/workflows/{README.md,child-usage.mjs,test-child-usage.mjs,SHA256SUMS}, examples/claude-native/CLAUDE.md and its generated copy adoption/new-wsl/claude-user-instructions.md (regenerated with tools/adoption/new_wsl_client_config.py --write-blocks), recipes/claude-native-ultracode.md, recipes/claude-native-profile.md, catalogs/foundation/{decisions.json,model-currency.json}, docs/model-currency.md, docs/decisions/2026-09-30-task-model-routing.md (one row, one overturn clause), docs/decisions/2026-09-29-sonnet-5-5-dispatch.md and docs/decisions/2026-10-07-model-currency-enforcement.md (one dated note each), evidence/receipts/haiku55-extraction-trial-20261007.json (new). The hot files manifests/stack.json and manifests/evidence.json are in the last commit, per the hot-file protocol.

SOTA sources

  • Claude Code 2.1.293 (anthropics/claude-code; npm @anthropic-ai/[email protected], published 2026-10-07T17:18:04.464Z, dist.shasum d653d8b46b676742dfd6dea808bfcc82f2fc0fd8, integrity sha512-PfPSvsd0zH1R1NhLKLebYdEXg/KcGCXc7LBntE186EWuZtBXa7sMnBqs0/uW/BvzkBEfHKqww/WbK/kMAk0TXg==): CHANGELOG.md line 5 at 79babc372d64101f981bd2b52c3dbe588596dc56 reads "Added Claude Haiku 5.5 (claude-haiku-5-5), now the default Haiku model on the Anthropic API — 1M context, $0.10/$0.50 per Mtok ($0.50/$2.50 for prompts over 100K)". That repository publishes no CI run at the tag.

  • Claude Code model configuration (version history, read 2026-10-07T23:58Z; the page carries no version):

    • "v2.1.293 | haiku resolves to Haiku 5.5 on the Anthropic API";
    • the "Earlier" row, where haiku resolves to Haiku 4.5;
    • the provider table: Bedrock, Google Cloud's Agent Platform, Microsoft Foundry and Claude Platform on AWS still resolve haiku to Haiku 4.5;
    • "Use v2.1.293 or later with Haiku 5.5".

    These are the source of the child-usage.mjs row, the README's version check and the Anthropic-API-only scope.

  • Claude Code sub-agents (choose a model): the per-invocation model parameter comes first in the model order, and haiku is an accepted alias. This is why the route is a per-call override, not a new agent type.

  • Claude Code advisor (advisor): subagents inherit the configured advisor, which is the source of the trial's advisor disclosure.

  • Claude Platform: the model-currency and stack.json rows rest on

  • Effort (effort doc): Haiku 5.5 supports all five levels, max included, and defaults to medium. Anthropic advises xhigh or max only where evals show a gain. This is the basis for the recipe's effort sentence, the "effort is the stage rule" note, and leaving out a claude-haiku-5-5 settings pin.

  • Trial design: the overturn condition recorded in docs/ultracode-token-routing-20260921.md and the workflows README: "a repeat trial in which Haiku returns no unanchored citation on two distinct extraction packets". It ran on the native Claude Code Workflow tool with this repository's source-scout and evidence-reviewer agents and child-usage.mjs.

  • Prices: LiteLLM model_prices_and_context_window.json (BerriAI/litellm main, ETag W/"35442afac19b8c20555fa17a4f487c7998c99b658444f04820628f1010f40a85", the copy ccusage caches). This is a third-party table and is labelled as one; its Haiku 5.5 base rate equals the CHANGELOG's $0.10/$0.50.

Evidence-class table

Claim Evidence class Command / receipt
On two distinct extraction packets Haiku 5.5 returned 8 of 8 runs exact (28/28 and 48/48 rows) with 0 unanchored citations, the same as Sonnet 5.5 native_proven receipt haiku55-extraction-trial-20261007 (workflow wf_aa532ddc-f32, Claude Code 2.1.293, one Mac)
A blind Opus evidence-reviewer found no defect in any of the 16 extraction runs (152/152 rows) native_proven (round 1 read from the run journal; round 2 also in the retained output) receipt data.runs[].blind_opus_verdict, data.blind_verdict_totals
The session's Fable 5.1 advisor answered 7 of 8 Sonnet and 2 of 8 Haiku extraction runs; the 6 unassisted Haiku runs were exact native_proven (transcript tool results and child-usage.mjs advisor iterations) receipt data.advisor, data.runs[].advisor_calls
Verbatim relay of a 2,511-character output: Haiku 5 of 6 byte-exact (the miss escaped all 360 double quotes), Sonnet 6 of 6 native_proven receipt data.c1_haiku_miss, data.summary
Every Haiku child resolved claude-haiku-5-5, recorded effort max and ran client 2.1.293 native_proven (the client's own per-message record) receipt data.identity_and_effort, data.runs[]
The retained answer-key scores reproduce local_integration (a deterministic re-run on retained outputs) python3 -I oracle.py <checkout at b16cb8cf> all-rounds.json <out> matched scores-all-rounds.json; re-run by the builder and by an independent verifier (cmp: identical)
Mean executor cost per child: P1 $0.0220 vs $0.2405, P2 $0.0180 vs $0.2132, C1 $0.0096 vs $0.1418. With advisor sub-inferences: P1 $0.2407 vs $0.6509, P2 $0.0180 vs $0.4672 local_integration (arithmetic on recorded usage times a third-party price table; not billing) receipt data.summary, data.prices_usd_per_token, provenance.cost_method
claude -p --model haiku, --model claude-haiku-5-5 and claude --bg --model haiku resolved claude-haiku-5-5 with exact replies native_proven (reported by the trial's probe children; not re-run here) receipt data.native_probes_reported_by_the_trial
haiku resolves to Haiku 5.5 from 2.1.293, and the background slot uses the default Haiku source_review (no read-back of the background slot) CHANGELOG@79babc37 L5; model-config version history
child-usage.mjs flags a haiku child that resolved an older Haiku under 2.1.293 or later synthetic (fixture rows; with the row removed, 2 tests fail) node test-child-usage.mjs

Local commands run

All commands ran on the final head e72e070f unless noted.

$ python3 scripts/validate.py
{"components": 69, "hashed_files": 10557, "profiles": 4, "receipts": 227, "status": "passed"}   exit 0
$ python3 scripts/validate.py --scan-file evidence/receipts/haiku55-extraction-trial-20261007.json
{"scanned_files": 1, "status": "passed"}   exit 0
$ python3 scripts/host_receipts.py validate                                     exit 0
$ python3 scripts/release_due.py --strict-if-repinned origin/main               exit 0
$ python3 scripts/validate_catalogs.py                                          exit 0
$ python3 scripts/validate_foundation.py --root . --json                        exit 0
$ python3 scripts/landscape.py --root .                                         exit 0
$ python3 blueprints/blind-catalog-convergence/audit_reports.py --check         exit 0
$ python3 scripts/validate_convergence.py --all-recorded --root . --json        exit 0
$ python3 tools/sota-convergence/build_verdicts.py --check                      exit 0
$ python3 scripts/component_matrix.py --check     {"rows": 32, "status": "checked"}             exit 0
$ python3 scripts/new_host_grand_list.py --check  {"status": "passed", "layers": 32, "winners": 66}   exit 0
$ python3 tools/sota-convergence/gap_crosswalk.py build --check                 exit 0
$ python3 tools/sota-convergence/gap_wave_ledger.py --wave gap-wave2-20260923 --wave gap-wave3-20260923 --owner gap-resolution --check   exit 0
$ python3 scripts/build_ecosystem.py --check                                    exit 0
$ python3 scripts/evidence_manifest.py --check                                  exit 0
$ python3 tools/adoption/new_wsl_client_config.py --check                       exit 0 ("check passed"; one pre-existing mcp-inspector warning)
$ (cd examples/claude-native/workflows && sha256sum --check --strict SHA256SUMS)   exit 0
$ node test-envelope.mjs            SUMMARY passed=254 failed=0 total=254       exit 0
$ node test-contract-mutations.mjs  SUMMARY passed=74 failed=0 total=74         exit 0
$ node test-child-usage.mjs         SUMMARY passed=174 failed=0 total=174       exit 0   (2 SKIP lines: no local tree-sitter-bash install; CI provisions it)
$ node test-usage-receipts.mjs      SUMMARY passed=18 failed=0 total=18         exit 0
$ node test-codex-envelope.mjs                                                  exit 0
$ node --test blueprints/native-skill-practice/test-contract.cjs   pass 6 fail 0   exit 0
$ python3 -m unittest   (full suite) started on e72e070f; its result was not recorded before this session hit its usage limit; CI runs it

The trial ran on 2026-10-07 from 21:31Z to 23:48Z (5:31 PM to 7:48 PM EDT). An independent three-stage Opus review covered the diff, the receipt against the private evidence, and a re-run of the checks. Its findings are folded in: the advisor disclosure and split costs, both rounds' verdicts, the portable instructions' stale line, the settings-pin and client-minimum notes, and the wording and number fixes. A second reviewer confirmed all 11 fixes and raised two items left open here: (A) catalogs/foundation/upstream-surface-dispositions.json:73-74 and :313-314 cite child-usage.mjs:67 and :1925, which the new comment line moved to :68 and :1926; (B) the receipt's P2 Haiku executor mean ($0.0180) and P1 Haiku advisor-inclusive mean ($0.2407) are means of unrounded per-child costs, so the rounded per-run values give $0.0181 and $0.2408, and provenance.cost_method should say so. A wording nit: README:1084 says "relays no command output verbatim" where CLAUDE.md says "relay none".

Decision record

The routing decision is recorded in these places:

  • The new "Haiku 5.5 for cheap exact extraction (2026-10-07)" section of examples/claude-native/workflows/README.md: evidence, boundaries, advisor, cost basis, settings pin, client minimum and overturn.
  • docs/decisions/2026-09-30-task-model-routing.md, the task-to-model routing record: one row and one overturn clause.
  • Decision lean-workflow-child-routing in catalogs/foundation/decisions.json.
  • Dated notes in docs/decisions/2026-09-29-sonnet-5-5-dispatch.md (the pin is left out on purpose) and docs/decisions/2026-10-07-model-currency-enforcement.md (the currency row).

Alternatives considered:

  • Keep every extraction on Sonnet. Rejected: the trial shows equal accuracy at under a tenth of the executor cost per child.
  • A new Haiku agent type. Rejected: the per-call override comes first in the documented model order, and it leaves the agent and role tables, which the envelope suite pins, unchanged.
  • Route command relay too. Rejected on the 5 of 6 relay result.
  • Add a claude-haiku-5-5 settings pin. Deferred: it would lift every Haiku session that names no effort without an eval.

Overturn: send extraction back to Sonnet when a routed Haiku stage returns an unanchored or misattributed citation that a later check finds, or when a haiku child resolves a model older than Haiku 5.5.

Host evidence

Not applicable: no file under evidence/hosts/ changed.

Merge interactions, follow-ups and limits

Checklist

  • New/changed GitHub Actions are pinned to a full commit SHA with a version comment (no floating tags). Not applicable: no workflow file changed.
  • New/changed workflows declare top-level permissions: {} and grant each job only what it needs. Not applicable: no workflow file changed.
  • No secrets are printed, logged or committed; no new required secret was added. The gitleaks pre-commit found no leaks in either commit, and the receipt passes the private-content scan.
  • No new paid hosting, subscription or billing surface was introduced.
  • Peer-owned untracked files and worktrees were preserved (not deleted, moved or overwritten).

🤖 Generated with Claude Code

https://claude.ai/code/session_01WhN8fzZs47tp6KunqRQ9gs

seathatflowsinourveins and others added 2 commits October 7, 2026 21:08
…trial

The 2026-10-07 repeat trial (workflow wf_aa532ddc-f32, Claude Code 2.1.293,
one Mac) met the overturn condition recorded for the "Cheap exact extraction"
row: on two distinct extraction packets Claude Haiku 5.5 returned 8 of 8 runs
exact with no unanchored citation, as Sonnet 5.5 did, and a blind Opus
reviewer found no defect in any of the 16 extraction runs. The session's
Fable advisor answered 7 of 8 Sonnet and 2 of 8 Haiku runs; the 6 unassisted
Haiku runs were exact too. Haiku's executor cost per child was under a tenth
of Sonnet's; its verbatim command relay was byte-exact in 5 of 6 runs.

- Routing: a source-scout stage may pass a per-call model 'haiku' at effort
  max for inventories and line-cited extraction that relay no command output;
  command running and output relay keep source-scout's Sonnet default, and
  judgment stays on Opus (workflows README section, recipe row and dated
  paragraph, portable CLAUDE.md and its generated new-WSL copy, profile
  recipe, task-model routing record, decision lean-workflow-child-routing).
  No claude-haiku-5-5 settings pin, on purpose (noted in the 2026-09-29
  dispatch record); the claude-code minimum stays 2.1.284 for now.
- child-usage.mjs: haiku alias row (Haiku 5.5 from 2.1.293, model-config
  version history) so a haiku child that resolves an older Haiku is flagged;
  tests for the boundary and the fallback; SHA256SUMS regenerated.
- Model currency: the background-haiku row moves to claude-haiku-5-5
  (release-note entry of 2026-10-07) and records the extraction route.
- Receipt evidence/receipts/haiku55-extraction-trial-20261007.json: derived
  counts, timings, prices, advisor use and file hashes only; the oracle re-run
  on the packets' source commit b16cb8c reproduced the retained scores.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WhN8fzZs47tp6KunqRQ9gs
…ation

manifests/stack.json: the Haiku model row moves from claude-haiku-4-5-20251001
to claude-haiku-5-5 (released 2026-10-07; Claude Code 2.1.293 default Haiku on
the Anthropic API), records the cheap exact extraction route and links its
qualifying receipt.
manifests/evidence.json: receipts[] gains haiku55-extraction-trial-20261007,
and files[] re-registers the files this branch changed that main already
lists, plus the new receipt (docs/lanes.md hot-file protocol). The component
matrix and new-host grand list regenerate unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WhN8fzZs47tp6KunqRQ9gs
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Oct 8, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant