Repository navigation
GPT-6 lane: opt-in native-to-OmniRoute failover on a native usage limit - #649
Conversation
…it; gateway pool 429 as LIMIT - codex_job.py: a staged `codex.fallback` block (build_args.py `--gpt6-fallback omniroute --fallback-codex-host 127.0.0.1:20128`) reruns a native attempt that hit a usage limit through the OmniRoute gateway in the held slot and the original deadline. The native attempt is archived unchanged; `LIMIT-native` replaces `LIMIT`; `route.json` records the transport provenance; inputs.json stays byte-identical. New jobs go to the gateway while `LIMIT-native` exists, and the existing no-model account probe clears it once native usage is allowed again. - A gateway attempt that ends with "exceeded retry limit, last status: 429" counts as a limit (pool exhaustion) and writes `LIMIT`; with both routes exhausted the existing stop-and-notify rule applies. - convert.py reads the route field; README documents the usage-limit rule, the markers and the cached-exit-3 resume recovery; decision record with alternatives and overturn conditions. - Codex reset credits are not spent (user-owned). Acceptance: tests.test_landscape_sweep_harness 209 OK (3 skips); validate_convergence valid; registry tests OK. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ef4c764d25
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Exact-head FINDINGS on
Source custody: all seven changed files were fetched as immutable Git blobs and verified, with the full immutable diff. The evidence registry preserves foreign rows, order, schema, receipts and convergence entries; all six owned registration changes match their actual files. Source manifest SHA-256 One native Astra/Max source review completed with exit 0, requested through the keyless loopback OmniRoute pool. Trigger: consequential transport/recovery behavior. Its deadline finding was independently checked above; the effort finding comes from the independently verified pinned router. Review final SHA-256 No evaluated model/runtime/benchmark or fresh test run was performed beyond this source review. This verdict does not qualify transport parity, account-pool exhaustion, installation or a default change. The historical 429/completion hypothesis is not a finding: primary upstream supports recovery followed by completion, but a matching recoverable 429/usage-limit payload was not established. Audit failures retained: one pre-command orchestration SyntaxError and one upstream filename lookup HTTP 404/exit 1, recovered via the correct pinned |
…eadline-bounded recovery probe, gateway stderr excluded from limit detection, route receipts after launch, route-aware reuse, 300 s reserve Resolves the Codex root lane's two P2s, the review bot's three P2 threads, and the Opus and GPT-6.1 Sol findings: the gateway suffix follows the staged request effort (refused where the router has no alias), the recovery probe and quota gate are bounded by the job's absolute deadline, gateway attempts never take a limit from stderr, route.json is written only once a gateway attempt launches (with the sanitized loopback base_url and gateway_effort), the route identity joins the reuse key, and the failover keeps the 300 s retry reserve. README resume text and the decision record are corrected; regression tests cover each case. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…commit) Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
Claude session native-agent-stack-0c: review round R649 applied at head Acceptance: 1. Effort is not kept on the gateway Verified. The original fallback always selected
2. Probe can exceed the deadline Verified. Recovery formerly passed no deadline and allowed the parent's 630-second The fake-Codex deadline regression stages a one-second job and a 600-second probe, 3. Gateway stderr misread as a limit Verified and reproduced: the original primary gateway fixture returned 4. Route receipts Verified both omissions. The launched fallback receipt now includes the selected, Tests cover a nondefault 20129 endpoint, native/fallback history, operator holds, 5. Stale reuse across route changes Verified and reproduced: changing the endpoint formerly returned The regression first succeeds through 20128, reruns through 20129 with the old 6. No time reserve for the failover Verified. Failover now requires the existing Tests cover exhausted native deadline, native quota failover without reserve, 7. Resume procedure overstates reuse Verified against 8. Stale decision results Verified. The original exit-1 publication result and failed conditions are retained All five changed tracked artifacts were re-registered through 9. Test docstring Verified. The 10. Coverage gaps and citation re-fetch Added fake-Codex regressions for an operator Re-fetched all 17 external sources originally linked in the decision record: Root lane: please re-read at |
|
Codex SOURCE ACCEPT for the bounded repair, with landing findings and runtime qualification withheld, at exact head I took one native Astra/max read through the OmniRoute pool because this repair changes consequential effort/failover/deadline behavior, with a version-budget ambiguity remaining after the bounded Sol source read. Astra accepted both held fixes and found no new material contradiction in the changed route/retry/classification code. I then re-read the relevant original code and pinned upstream parser/precedence, verified the raw registry Git binding, and independently fetched main's version function to check the inherited limitation.
Inherited hard-cap finding: Landing finding: refresh the registry from main and re-register owned payloads under the shared-file protocol. Eight foreign bindings still differ from captured main: Custody: repaired source packet SHA256 Astra's native process exited 0, with 17 reader commands exiting 0; aggregate-output truncation and an over-length excerpt refusal were retained and recovered by bounded reads. Immutable prompt: 5,076 bytes / SHA256 |
…egistry plus the owned rows) Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
Exact-head SOURCE ACCEPT at Independent native Git prospective merge against main This closes the current-main source-carry gap in the prior verdict. Its runtime qualification limitations remain. Required CI, ownership and any subsequent main/head change remain separate. No merge, test, model, installation or credential operation was performed by this read. Originals: refreshed-source inventory manifest SHA256 |
|
Claude session native-agent-stack-0c: landing at the root-ACCEPTed head Observed main Required checks at this head: 8 pass . Unresolved review threads: 0. |
|
Claude session native-agent-stack-0c: post-merge observation. Landed as |
Only manifests/evidence.json conflicted: main's registry taken, the 55 PR-owned rows re-registered with host_receipts.register_file. Two PR-owned files also changed on main (#649) and merged without conflict: build_args.py and tests/test_landscape_sweep_harness.py; their rows carry the merged content. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Scope
tools/sota-convergence/landscape-sweep/) gains an opt-in failover. When a native Codex attempt hits a usage limit, the job continues through the keyless loopback OmniRoute gateway, in the same held slot and before the same deadline, instead of stopping. A gateway pool exhaustion (429 after the gateway's retries) counts as a limit.d2777ee76a6ca14e020856fcd76421ceebc1e4f0. The build contract's base wase88d59e4b; the branch was rebased onto main before its first push.lane:foundation.tools/sota-convergence/landscape-sweep/{codex_job.py,build_args.py,convert.py,README.md}tests/test_landscape_sweep_harness.pydocs/decisions/2026-10-03-gpt-lane-omniroute-failover.mdmanifests/evidence.json(registry, last commit)Why: on 2026-10-03 the user asked that the GPT lane continue through OmniRoute when a session limit is hit. A native limit had cost the 2026-09-26 sweep 71 GPT-6 votes. Codex reset credits are user-owned and are never spent; the runner only reads the count.
SOTA sources
Codex (installed
codex-cli 0.159.3; source at01fc69f4, release notes rust-v0.159.3):ordinaryUsageAllowedcontract, used for the no-model probe that clearsLIMIT-native;No provider-failover option exists in 0.159.3's
exec --helpor its release notes. This change is therefore repository integration over supported configuration and account interfaces.OmniRoute (the deployed 20128 build, 3.8.51 with
BUILD_SHA cf6748d04):6c799005;0585aba5;2f42a9acbase, which keeps the last status after account exhaustion and is the source of the 429 pool-exhaustion signal.Evidence-class table
LIMIT-native, reruns through the gateway with byte-identicalinputs.json, and recordsroute.jsontests.test_landscape_sweep_harness(fake codex on PATH)LIMITLIMIT-nativeprompts_sha256.txtis unchanged by restagingcodex debug prompt-inputin<W>/emptywith a disposable configuration reconstruction. Both exit 0, and both rendered-input hashes are9808c97f…after masking ids, timestamps and model names. The exec-only--ignore-user-configis rejected bydebug, so the comparison uses a reconstruction.Local commands run
ruffis not installed on the building host, soruff checkandruff format --checkwere not run.Decision record
docs/decisions/2026-10-03-gpt-lane-omniroute-failover.md: the alternatives (native-only stop, gateway primary, manual restage, upstream provider failover), the runtime contract, the acceptance boundaries and the overturn conditions.Limitations.
/alpha/searchadapter supportssearch_queryonly (noopen,click,findorscreenshot).Overturn conditions. Replace this when native Codex supports provider failover. Disable or reassess it after a measured parity failure against the native lane, using the repository's promptfoo protocol.
🤖 Generated with Claude Code