Skip to content

pi trial record: pi v0.99.1 behind OmniRoute with the token-save practice (S1, S2, R1 observed, decision trial) - #524

Closed
seathatflowsinourveins wants to merge 3 commits into
mainfrom
claude/pi-omni-trial-20260929
Closed

seathatflowsinourveins wants to merge 3 commits into
mainfrom
claude/pi-omni-trial-20260929

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

  • What this PR changes: adds the dated record of the pi trial through OmniRoute (blueprints/convergence-practice/pi-omni-trial-20260929/: README, experiment.json, the local kit, sanitized per-run evidence, two gateway read-backs and pool readings) and registers its 28 files and the record in manifests/evidence.json. pi v0.99.1 ran 28 real tasks on sharedgw/gpt-6.1-sol through the 20129 lane (S1 wiring smoke 3, S2 24 = 3 tasks x 4 arms x 2 attempts, R1 one run killed at its second response and resumed); all passed their fixture checks. On these small fixtures the token-save stack used more tokens than plain pi (about 4.9x the uncached input), the gateway lane removed 192 of 413,153 input tokens, and its effect on uncached input is unresolved. Decision stays trial; nothing is adopted, and no verdict row, catalog, pin, client setting or gateway setting changes.
  • Base commit: b4056a355580f079560cb612cf03a0a2568c3af7 (rebased onto main after OpenHands recipe lock: PyJWT 2.14.0 relock closes ten advisories (unblocks the required osv-scanner check) #525, the PyJWT relock, and OmniRoute rebuild 2026-09-30: decision record, evidence artifacts, owner-record script, unit templates (records what runs; compression on 20128 not applied) #530, the OmniRoute rebuild record it cites)
  • Lane: lane:foundation
  • Owned paths touched: blueprints/convergence-practice/pi-omni-trial-20260929/**; manifests/evidence.json (shared hot file: last commit, taken from main and re-registered after the rebase, per docs/lanes.md; the re-registered manifest is byte-identical to the auto-merged one).

SOTA sources

Evidence-class table

Claim Evidence class Command / receipt
28 real pi tasks passed on sharedgw/gpt-6.1-sol via 20129 (S1 3, S2 24, R1 1) native_proven (this host, 2026-09-30) evidence/{s1-runs,s2-runs,r1}.json, experiment.json observations
Plain pi cost fewer tokens than the stack arms on these fixtures; the lane removed 192 tokens synthetic (three small fixtures, n=2 per cell) evidence/s2-runs.json, evidence/s2-summary.txt
R1: SIGKILL after the second response, pi --continue finished and passed, no descendant survived native_proven (one run) evidence/r1.json
Gateway state during the runs (engines, output styles, session-dedup and lite settings, 20128 off) native_proven read-back evidence/gateway-readback-20260930T0159Z.json (first apply: ...T0009Z-first-apply.json)
Upstream mechanics (lossy request policy, headerless plan, documented lane, shipped default off) source_review OmniRoute 2f42a9ac1 files above
96 offline probe lines pass, 4 KNOWN-FAIL, 0 FAIL local_integration kit/probes.py, kit/stage.py --check

Local commands run

$ python3 scripts/validate.py
exit 0  {"components": 69, "hashed_files": 8378, "profiles": 4, "receipts": 174, "status": "passed"}
$ python3 scripts/validate_convergence.py --all-recorded
record[0..25]: valid
$ gitleaks dir blueprints/convergence-practice/pi-omni-trial-20260929
no leaks found
$ (host-string scan of the trial directory: host user name, /home/, /tmp/, session id) 0 matches

Independent reviews, two families, one repair round each: (1) an Opus evidence-reviewer (read-only): no blocker, 9 should-fix, all applied (the "upstream-default" label corrected to "upstream's documented safe lane plus ccr", cache-warmth confound disclosed, unarchived figures removed or labelled, the 14:12Z run's role and compression note corrected, read-back extended to the full key set). (2) A GPT-6 astra review at max through OmniRoute (20129 to 20128, compression off, read-only sandbox): no blocker, every recomputed number confirmed (S2 table, request-2 shares, paired differences, R1 sums, all 29 usage objects, 47 hashes); about 11 should-fix and 10 minor findings, applied in the second commit: analysis scripts (summarizer merged the gateway-off control into stack, no stage selection, null usage read as zero, silent row truncation, R1 export overwrite, run exit statuses masked), the mutant self-test dropped verdicts from the probe tally ("of 98" is now "of 100"), wording scoped to what n=2 fixtures support, and two undisclosed confounds now disclosed (stack and stack-ext shared Serena/jCodeMunch state directories; the header-less arms passed through the gateway's lite whitespace fold). Not changed, by design: the runners and templates that produced the evidence (their hashes must match the runs); their findings (inherited PI_CODING_AGENT_DIR, PID-by-PID kill) are disclosed in the record. Residuals: no live re-run of R1 with the hardened kill; provenance of the executed run_s2.sh predecessor.

Decision record

The record's own decision is trial (experiment.json); its next_decision_changing_test is a large-output fixture (S2b) with the same four arms.

Host evidence

No files under evidence/hosts/ change.

Checklist

  • New/changed GitHub Actions are pinned to a full commit SHA with a version comment (no workflow changes).
  • New/changed workflows declare top-level permissions: contents: read (no workflow changes).
  • No secrets are printed, logged or committed; no new required secret was added without a documented owner.
  • No new paid hosting, subscription or billing surface was introduced.
  • Peer-owned untracked files and worktrees were preserved (not deleted, moved or overwritten).

🤖 Generated with Claude Code

@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 30, 2026
@seathatflowsinourveins
seathatflowsinourveins force-pushed the claude/pi-omni-trial-20260929 branch from f260d4b to 3443ee3 Compare September 30, 2026 04:01
…tice (S1, S2, R1 observed, decision trial)

Kit, four-arm S2 (24 runs), S1 wiring smoke (3 runs) and R1 kill-and-resume on sharedgw/gpt-6.1-sol via 20129 after the gateway owner's corrected token-save configuration; sanitized per-run evidence, gateway read-backs and pool readings; an independent review's findings applied.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…tra max via OmniRoute)

Kit: label aggregation, --tag selection, null-safe shares, missing-row and truncation flags, R1 export guard, mutant self-test tally, run exit statuses; evidence regenerated from the same call logs (totals unchanged). Record: wording scoped to what the fixtures support, shared MCP tool state and lite whitespace folding disclosed, provenance of the kit files stated.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…on (hot-file step, base b4056a3)

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

Takeover notice from Claude session native-agent-stack-0c, objection window until 2026-10-03T06:36Z.

A verified triage (Sonnet facts plus an Opus judge, re-read against main 56473e4b8) found this record is still valuable, not on main, not superseded by any merged PR, and has no live owner. After the window, if no one objects, this session will bring the branch up to current main:

  • merge main, with no force-push;
  • re-register its files through the docs/lanes.md hot-file protocol;
  • correct statements that went stale since it was opened, using dated notes, with evidence and frozen inputs unchanged;
  • obtain one independent read-only review, then land it.

To keep or land it yourself, comment here before 2026-10-03T06:36Z.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants