Repository navigation
Organic-invocation E2E v1.1: protocol, pilot spec and the promptfoo-driven harness (run of record smoke-20261006c) - #786
Closed
seathatflowsinourveins wants to merge 42 commits into
Closed
seathatflowsinourveins wants to merge 42 commits into
seathatflowsinourveins wants to merge 42 commits into
Conversation
…e promptfoo-driven harness Adds evidence/artifacts/organic-e2e-20261005/: the delivered protocol v1.1 and pilot spec (home prefix written as ~ for the publication scan; the delivered bytes are hash-referenced in each file's header comment), and the harness that implements the pilot spec on promptfoo 0.123.1 exec: providers: - prepare.py (stage 1): fixture from git archive 9e95532, stripped and gated, setup files from local git objects, deterministic tar, D oracles in a scratch extraction, routing-registry candidates, R2 lint, per-trial Claude settings (8.1 deny list), Codex rules (codex execpolicy check per rule), clone gates, S7 baseline, meter, quota gate, seeded promptfoo configs per cell. - launcher.py: the exec: launcher contract (ledger rows, NEUTRAL_ROOT fixture, per-trial clone or settings, shared flock with the meter read inside it, clean login shell, stream tail with the 9.1 kill rule, nested-client watch, draft copy and fixture manifest, always exit 0 with one JSON line). - block.py, collect.py, grade.py: gates around one eval, stage-5 collection (Loki, rollouts, skill_usage --lanes --call-ledger, gateway call logs joined on client_metadata.thread_id, agentsview), stage-6 grading (joins, stream and Loki agreement, marker scan, R1 provenance tags, watcher, OIR with Wilson intervals) and the stage-0 replay over the v1 captures. - sdk_claude.py (CL6) and sdk_codex.mjs (CL7): written, not yet run. Co-Authored-By: Claude Opus 5.5 <[email protected]>
- prepare.py --prompted builds prompted cells (prompted-<base cell>) on lane organic-e2e-prompted with their own fixtures, as R10 requires for probes, canaries and oracle runs. The stage-2 canaries of the pilot spec are in harness/stage2-canaries.json. - grade.py keeps prompted lanes out of OIR. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…actor scope Findings from the first live smoke (run smoke-20261005b) fixed in the grader: - Every call, result and hook output carries its event index (Claude) or tool ordinal (Codex, shared by the exec stream and the rollout), so a block only tags uses that come after it. - Hook blocks are the hook's own output (additionalContext, systemMessage), never the hook event's input; ai-memory and handoff injections are store blocks, other tool-native hook context is hook-nudged. - Store and hook blocks, and consulted skill bodies, apply within one actor; a subagent or child thread has its own context. - An item's own output (ctx_search text naming ctx_* tools, a ctx_* wrapper around an auto-memory read) does not direct that item; plain-English aliases such as "search first" no longer name an item in a block. - A consulted skill's body is its SKILL.md, not a command output that may hold several skills. - RTK PreToolUse rewrites (updatedInput) of main-stream Bash calls become hook-rewritten rtk uses; S6 reports the rewritten share and hook sources. - G3 also checks child-transcript MCP and Skill calls; calls killed before a result agree only when Loki has none either. - G13 does not count reads of the trial's own auto-memory directory (exempt, logged as own_auto_memory_reads); the env arm's harness text is the text the client loaded. - prepare.py records MCP tool names by server for the Codex tagger. Co-Authored-By: Claude Opus 5.5 <[email protected]>
- harness/pilot.py runs the pilot spec's stage order as one command: the stage-0 replay (G1 must hold), prepare, the stage-2 probes and canaries and the gate-0 G1 trial, then the Codex blocks C1-C5 and the Claude schedule in parallel chains, collect and grade. It sequences block.py calls only; each block keeps its own gates, and a refusal stops that chain. - When a host harness file no longer matches the protocol's hashes, one Claude probe is added and prepare.py --skip-tests drops G1|claude-sdk (the give-way rule), so the Claude total stays 14. - stage2-canaries.json adds the two Codex probes; the marker scan, not the prompt, decides each probe. - Exercised end to end with stand-in clients (17 trials, no model call). Co-Authored-By: Claude Opus 5.5 <[email protected]>
CL7b (promptfoo's openai:codex-app-server) has no launcher and no exec stream. collect.py now takes its thread from Loki (env = trial_id), copies the thread's rollouts, copies ./draft/ and hash-manifests the fixture, as the protocol's CL7b entry assigns to stage 5. grade.py reads that thread's rollout as the primary source when no exec stream exists and records which source it used. Written from the observed shapes; not yet run against a live CL7b trial. Co-Authored-By: Claude Opus 5.5 <[email protected]>
A process listing shows a client's argv to the model, and the smoke's launch lines carried run-root paths that name the experiment (the per-trial settings file, the -o file and the CODEX_HOME clone). R2 (f) covers every launch string the model can see, so prepare.py now creates a neutral per-run directory (~/.cache/wsr/<8 hex>) for settings, prompts, -o files, clones and the SDK launchers (bin/run.py on the base interpreter with the SDK venv first on sys.path, bin/run.mjs, bin/sdk), and records it as run.json trial_root. Records stay in the run root. A stand-in self-test confirms the launch lines keep no run-root path; the protocol's own lane tag (ecosystem.lane=organic-e2e in OTEL_RESOURCE_ATTRIBUTES) is the remaining experiment word in a client's environment. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…enance) Fixes the independent verifier's findings on 8d43ec6f: - GitHub credential (finding 6): the Codex clone adds GH_CONFIG_DIR to every stdio MCP server's env as well as the shell policy, gated by a parsed diff of config.toml; the CL3 line, the CL7 launcher and CL7b's cli_env carry it too. A second gh canary runs through ctx_batch_execute. - Stage 2 is a gate: pilot.py runs the Codex probes and canaries, the gate-0 G1 trial (its own cell) and the Claude probe, then collect.py on those trials and grade.py gate0; stage 4 refuses unless gate0.json passes. - Claude completion: run.json claude_completion (censor-at-T as written, or complete-at-result with a grace) and any T change need an amendment reference; the launcher records the result event's arrival, time-to-result and post-result time, and the grader splits spend before and after the result. The Claude chain refuses while the policy is undecided. - Session cap: the launcher counts launched Claude sessions inside the lock and refuses at 14; stage 1 refuses a larger Claude schedule; resume runs only tests short of their repeat; a pre-launch meter or lock refusal writes DEFER.claude and the test is carried forward. - Rollouts are found by thread id across local and UTC date folders; stage 0 reads the v1 captures' Loki rows over their own window, saved once into the coordination tree. - Grader: child transcripts take their actor type from their meta files, Workflow calls are agent-type uses, spawn prompts and Codex forks are provenance sources; RTK rewrites come from transcript hook attachments joined by tool_use id over every actor; hook text is classed tool-native or harness by command; tool results carrying a harness marker tag later uses harness-read; G13 also covers host checkouts, user-level harness files, trial roots and other fixtures; a process-table read detector; task-induced reads the call's input; Claude validity needs the transcript. - Neutral argv: opaque cell codes and test refs, configs, shims, runner link and interpreter links under the trial root, the pilot re-executes through bin/p.py, and the launcher lints every ancestor's argv before a trial starts. - Gates: G10 (SDK parity) and G12 (D oracles recomputed in a fresh extraction at stage 1) are implemented; G7 needs the canaries; G11 needs the forwarded effort; G6 reads each trial's own first in-stream reading, and the launcher applies the prior to it. - Minor: S7 before-snapshot after the lock, the currency due file in S7, prompted entries on a base cell no stage runs are refused, and the registry takes a reviewed list (--registry-review); organic trials refuse while it is provisional unless a development run records an allowance. Exercised with stand-in clients (selftest-fix-b) and synthetic grader checks; the live 4-cell smoke follows. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ows the run's T The clone's new env tables for MCP servers that had none keep Codex's default variables: openai/[email protected] codex-rs/rmcp-client/src/utils.rs:16-39 adds a server's env to DEFAULT_ENV_VARS (HOME, PATH, :163-166). CL7b's turn_timeout_ms now comes from the run's T instead of the constant. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The live smoke-20261005c stage 2 showed the launcher's post-exit S7 snapshot racing Codex's own marketplace upgrade, which writes temporary git objects into the per-trial CODEX_HOME clone after the client exits: the walk hit a vanished file, the exception escaped and the trial was left without an exit row (gate 0 then failed it, as it should). The tree manifest now records a vanished or locked file as unreadable, post-exit bookkeeping failures are written into the exit row, and G4 requires every launcher trial's own host comparison instead of defaulting a missing one to equal. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The live smoke-20261005d stage 2 stopped on a host view that one trial's before-snapshot caught mid-write: Codex sessions re-sync the remote curated plugins into the shared ~/.codex/plugins/cache at start (the clone symlinks plugins, protocol 4.3), and with three trials in flight another session's sync was half done. The listing was back at the stage-1 baseline after the trial. Trial and block snapshots are now read until they hold still, a STOP needs a view that differs from the baseline after the trial (or a new trust entry), and a before-only difference is recorded as a transient. G4 and gate 0 use the same judgement. Co-Authored-By: Claude Opus 5.5 <[email protected]>
pilot.py --from-stage 2 --rerun-gate0-failures runs once more each stage-2 test whose gate-0 check failed (a Claude probe stopped by the meter goes in the next 5-hour window), then collects and checks gate 0 again; DEFER.claude is cleared at the start of stage 2 as well as 4 once the newest meter reading allows a start. Co-Authored-By: Claude Opus 5.5 <[email protected]>
prepare.py and pilot.py accept cell@item_id:INSTANCE beside the global item_id:INSTANCE pairs, so a smoke can run CL7 and CL7b on G1 without adding a Claude session. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Codex and Claude chains run in parallel in stage 4, so a trial's snapshot can meet another session's plugin sync; up to eight reads, two seconds apart, before a view counts as unsettled. Co-Authored-By: Claude Opus 5.5 <[email protected]>
… in both Codex arms G10 (SDK parity) now holds only when CL6, CL7 and CL7b were each exercised and passed: a cell with no graded trial is listed under not_exercised and the gate reads None, never a pass; a failed check still fails it. The stage-2 ctx_batch_execute gh canary also runs in the env clone (canary-gh-auth-ctx-env on codex-env), the arm where the verifier observed the host gh login (finding 6); gate 0 and G7 require it. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…L7/CL7b take the omniroute profile as --config
The live smoke-20261005f exposed two launch defects.
- The claude-native trial was killed as a nested client when the model
ran `find / -xdev -type f ...`: Claude Code's native binary is
multi-call (run as argv[0] bfs, ugrep or rg it is the embedded find,
grep or ripgrep; checked on 2.1.290 with exec -a), and the bfs
process's `-type` passed the print-flag test. A nested Claude session
now also needs argv[0] to name the client; an exe beside the trial's
binary in the versions directory counts as Claude, and any exe named
codex as Codex, so a client updated mid-run is still seen.
- CL7 and CL7b exited at start: codex-cli 0.160.0 refuses
`profile = "omniroute"` through --config ("legacy ... no longer
supported; use `--profile omniroute`"), and neither the Codex SDK
0.160.0 nor promptfoo's codex-app-server provider can pass --profile
(`codex -p omniroute features list` also says --profile does not
apply to app-server). Stage 1 now freezes the profile file's keys,
minus model and model_reasoning_effort that every launch sets, as
run.json codex_profile_layer; CL7 passes them as the SDK config
object and CL7b as cli_config. `codex features list` with those -c
pairs loads them (standalone_web_search true), and the legacy key
fails to load. This settles the protocol's [nv] on CL7's profile.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
…read at max effort The live devcheck-cl7-20261006a run (CL3 reference, CL7 and CL7b on G1 at ae032cf9) passed joins, G3 and the Codex half of G10, with two parity gaps on CL7b: its launched row had no gateway build, so G11 could never hold for it, and codex.conversation_starts reported the host config's ultra because the provider sets effort per turn only. block.py now records the build on CL7b's launched row (the grader reads it there when the pre-launch row has none), and CL7b's cli_config sets model_reasoning_effort max as CL3's -c does. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…rried forward Finding 4 left one path consuming a Claude test without an observation: the launcher writes the launched row before the client starts, so a trial it kills at its own first in-stream rate_limit_event (the 9.1 prior, when no reading 30 minutes old existed at the lock) counted as run, and resume never ran its test again. pilot.py now carries such a test forward like a pre-launch refusal; the session still counts toward the cap of 14, because the launcher counts every launched Claude session. The grader keeps these trials out of the session-content gates (G2, G3, G5, G6) and lists them; G4, G7, G8, G13 and G14 still apply. Checked with stand-in clients (selftest-h at 16ff012b plus this change): the first Claude trial was killed at 1.0 s with meter_prior_first_event, DEFER.claude stopped the chain, the resume cleared it and ran the carried test again, and both Claude trials then ended by the complete-at-result grace kill (result at 3.1 s, killed 5.1 s later, not censored); every host trust view equalled the stage-1 baseline. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The chrome-devtools MCP server joined both clients' MCP lists after the protocol was written (it is pending at Claude init and configured as a stdio server in the Codex config). It runs outside the Codex sandbox and reaches the network, and R8 has no pre-execution rule for it. The watcher now logs each of its calls in both clients (never a halt), so a trial's browser use shows in the report. A pre-execution deny would be a treatment change, so it stays a decision for the command center. Co-Authored-By: Claude Opus 5.5 <[email protected]>
G15's "verified negative" was always unset, so the gate could never show it. R4 makes G1 to G6' should_not for every item, so a valid trial on a control task whose joins and stream-Loki agreement hold checks its non-use of every item against the raw sources. G15 now counts such a trial as the verified negative and lists, per trial, the items it did use: those are should_not uses, a measurement rather than an instrumentation failure. Regraded in memory: smoke-20261005f's gate-0 G1 trial and devcheck-cl7-20261006a's three G1 trials qualify, and each shows a context-mode use on G1. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ells The pilot spec's stage 3 (ten prompted oracle runs on Codex) had a mechanism (--prompted) but no inputs, and the entries would have joined the stage-2 prompted cells: they ran before gate 0 was checked, gate 0 then required each of them to complete (a censored oracle run, which only drops a positive, would have stopped stage 4), and an entry's task_id and instance replaced the organic task's prompt in tasks.json, so the grader would have tagged that task's organic uses explicit. - prepare.py: a prompted entry takes "stage" 2 (probes and canaries, cell prompted-<base>) or 3 (oracle runs, cell oracle-<base>); keys must be unique; a prompted task's own id is always prompted/<key>, and an entry's task_id and instance are kept as oracle_for. - pilot.py: stages 3 and 4 both need gate0.json passing; stage 3 runs each oracle cell as one block before stage 4. - stage3-oracles.json: the pilot spec's ten oracle runs (jcodemunch.P1, headroom.P2, semble.P1, socraticode.P1, context-mode.P1, search-first.P1, skill-creator.P1, variant-analysis.P1, diagnosing-bugs.P1, ast-grep.P1) on codex-native with C1's sandboxes, each the organic prompt after one line naming the item (the MCP server, $skill or sg). Frozen with the harness at stage 1. Checked with stand-in clients (selftest-j): gate 0 graded the eight stage-2 trials only, stage 3 refused while gate 0 failed, and with a forged passing gate0.json it ran the ten oracle trials on lane organic-e2e-prompted, one attempt each. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…te 0 The previous commit moved the gate-0 check ahead of stage 3, where it also refused a resume at stage 5 or 6 that only collects and grades. The check now applies only when stage 3 or 4 sessions would run. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…behind its header) The live smoke-20261006a CL6 trial (Claude Agent SDK, jcodemunch.P1) exited after 1.0 s with rc 1: prepare.py writes bin/run.py as a two-line sys.path header (the SDK venv's site-packages, kept out of argv because its path names an experiment word) followed by sdk_claude.py, whose `from __future__ import annotations` must be the first statement, so the file was a SyntaxError. That was CL6's first live run; the self-tests stood in for the CLI, not for this file. sdk_claude.py drops the future import (it needs none on Python 3.13), and stage 1 now compiles bin/run.py and refuses the run if it does not compile. Checked: header plus sdk_claude.py compiles, imports claude-agent-sdk 0.2.163 from the venv's site-packages (its ClaudeAgentOptions has effort and session_id) and parses its argv. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ted one smoke-20261006a's G12 failed on context-mode.P1: stage 1 reused the fixture template that a self-test (prepare.py --skip-oracle-tests) had built a few minutes earlier for the same builder hash, so the frozen D oracle for that card was null, and G12's recomputation with the tests differed. A template built without the oracle tests now has its own cache directory (suffix -nt), records oracle_tests_run, and is reused only by a run with the same setting. Co-Authored-By: Claude Opus 5.5 <[email protected]>
smoke-20261006a stopped mid-run when another lane edited ~/.codex/omniroute.config.toml at 05:01Z: the only change was the profile's default model (cx/gpt-6-astra to cx/gpt-6.1-sol), which every Codex cell overrides (CL3 -m, CL7 and CL7b model; the frozen CL7 and CL7b profile layer already omits it). Protocol S7 does not list the profile; the harness had added its raw bytes to the gated view. The gated view now holds the parsed profile without the keys each launch sets (model, model_reasoning_effort; one constant in common.py that prepare.py's profile layer uses too), and the raw file hash moves to an ungated info record. The 30-minute settle rule still reads the file's mtime, as section 9.1 words it. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
…arrives before T smoke-20261006c's claude-native trial (search-first.P1) started a background Workflow and ended its turn at 576 s with "I'll write the recommendation when it reports back". Claude Code in print mode held the result event while the workflow ran (task_progress events until the end) and wrote it only on the timeout's SIGTERM, at 900.7 s, with duration_ms 575920. The launcher's complete-at-result rule compared the result with its own kills only, so the trial was recorded complete (rc 124, censored false). - launcher.py: a result counts only if it reached the stream before T (result_before_t); one written at or after T censors as timeout_after_result. - grade.py: effective_exit re-checks the rule on every exit row, so a run whose frozen launcher predates it is graded the same way, and the record names the override (grader_override). Unit-checked: result at 900.7 s with T 900 -> timeout_after_result; result at 524 s then the grace kill -> complete; no result -> timeout. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…eway entry is best-effort Two G3 and G10 failures in the live smoke-20261006c were artifacts of how a trial ends and of the gateway's known logging race, not of the instrumentation: - G3: the claude-env trial's ctx_batch_execute started at 10:30:06Z and was still running when the timeout ended the trial; the transcript then holds a "Connection closed" result written at 10:43:17Z, as the client exited, and Loki has no row for it. A call whose result was written at or after the trial's termination (the launcher's kill time, or launched time plus T for rc 124) is now marked terminated: never a completed use, and unresolved in G3 (agreement.unresolved_terminated), like a call with no result. - G10: CL7b's three /v1/responses rows in the gateway call log have status 0 and their detail returns 404, the unfinished rows of a client that exits right after response.completed, so no thread join is possible. The command center's telemetry caveat (item task-ns2604-coop-20261005T200956Z) makes call logs best-effort and client-side telemetry the source. G10 now rests the route on the rollout's omniroute provider, records the joined gateway entry as gateway_entry_best_effort, and lists a missing one as a G10 gap. Regraded in memory with this grader: smoke-20261006c passes G2-G10, G12 and G14; G11 fails (forwarded effort not exposed) and G13 fails (env-arm reads of host checkouts), both with their recorded gaps. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
added
the
lane:foundation
seathatflowsinourveins
force-pushed
the
claude/ns2604-organic-harness-20261005
branch
from
October 6, 2026 11:33
7dc41ec to
5085150
Compare
…k-ns2604-coop-20261006T105529Z) AMENDMENT-v1.1-20261006.md records decisions 1-8 with the item id beside the protocol, which stays verbatim; the harness now follows them: 1. Censoring: T = 1,800 s and complete-at-result by default; each Claude trial records its final turn end and its result event; no result by T writes a no_result_diagnosis and holds the cell (HOLD.<cell>) for the operator. 2. Exposure: a CLI is exposed only through a native surface its own upstream installer put in place (mineru's skill, worktrunk's plugins); PATH-only targets stay out of OIR (not_exposed_path_only). 3. G13: host-checkout and other reaches are tags with their own OIR stratum; only gold or answer-source reads invalidate a trial or fail G13. 4. chrome-devtools: watcher-only logging, no pre-execution deny. 5. G10: the gateway entry is best-effort; fast-exit undercount recorded. 6. Meter: expected usage against remaining headroom (resetsAt honoured), no account-wide kills; rate-limited trials are marked and re-run. 7. One in-run re-baseline per run, never for trust-bearing keys, the Claude settings file or the arm-defining files; the trials in flight (one Claude trial, up to -j trials of the Codex cell, a straddled CL7b trial) are re-run; a change seen only at a block's end, or any second change, still stops it. 8. RP4: only the forwarded effort fields of pipeline details are kept; the operator's on/off record and the observed state are reported. Decision 9 (gateway web-search wiring) is a host change and is not here. Checked offline only: module imports, a symtable undefined-name scan, 56 synthetic helper checks and a read-only re-grade of smoke-20261006c. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Oct 6, 2026
…k-ns2604-coop-20261006T132948Z) and the 1f81d64 P3 The command center decided at 13:29Z that a classifier of the commands a model typed cannot converge, so G13 is now structural (harness/isolation.py): 1. Every listed answer source is hidden from each trial's mount namespace. The launcher (every CLI and SDK cell, the prompted and gate-0 cells) and block.py (CL7b, through the provider's codex_path_override, which points at a per-attempt wrapper) run the client's process tree under bubblewrap 0.11.1 (/usr/bin/bwrap, unprivileged). Empty tmpfs: the coordination state (run roots with oracle runs, other trials' drafts and grades; the suite cards; the v1 captures) and the fixture cache (oracles.json). Empty tmpfs with the trial's own entries bound back: ~/.cache/ws (own fixture), ~/.cache/wsr (own prompt, settings, clone, -o file, the run's neutral binaries) and ~/.claude/projects (projects outside the experiment, and the own one). A fresh per-trial folder over the shared Codex sessions folder (the clone's alias), Claude's file history, context-mode's per-project stores and Claude's /tmp area; the trial's rollouts are published to the native sessions folder after it exits. The native home and client configuration stay visible and writable, the system runtime is read-only, the network and environment are unchanged. Options pass through --args (a memfd), so the namespace's PID 1 shows none; no --new-session, so the launcher's TERM reaches the client (it signals the tree below the wrapper, then KILLs the group after 30 s). systemd's InaccessiblePaths= fallback (systemd 259) is not needed: bwrap hosts every launch path. 2. G13 checks what was hidden (isolation.check): the launched row's receipt (wrapper argv, mount operations, hidden list with path and content sha256s) covers every listed location, nothing is bound back into a hidden root but the trial's own entries, the client tree ran in its own mount namespace (--info-fd), no sampled process was in the host's (nested sandboxes are counted apart), and stage 1's wrapper-only self-test passed (isolation-selftest.json; prepare refuses on a failure, gate 0 requires it and each stage-2 trial's receipt check). A trial is valid only if its own check passes. 3. The optional fanotify audit stays documented: a pid filter needs CAP_SYS_ADMIN (fanotify_init(2)), and the harness runs no sudo. 4. The command classifier (grade.reach) is a diagnostic tag only, reported under G13's classifier_diagnostic. The GPT micro-check of 1f81d64's F1 and F4 P2s concern it and are recorded as superseded. The P3 of that micro-check: the deadline and completion decisions (the hold, result-before-T, the censoring reason, the grader's effective_exit and its no-result list) read the unrounded offsets (duration_exact_s, time_to_result_exact_s, common.decision_times); the one-decimal fields are presentation only. Checked offline, no model call: harness/test_isolation.py, 23 of 23 (on the harness at 1f81d64, 3 of its 17 runnable tests fail and 14 error); the wrapper-only self-test, 43 probes hidden with ENOENT and 7 client start checks (an S7 snapshot before and after it is equal); a symtable undefined-name scan and imports; and a read-only re-grade of smoke-20261006c: every gate matches the 10:48Z grade, G13 failing for want of receipts (that run predates the wrapper), 0 of 16 trials valid. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
force-pushed
the
claude/ns2604-organic-harness-20261005
branch
from
October 6, 2026 14:27
1f81d64 to
9d4214e
Compare
…k-ns2604-coop-20261006T132948Z) and the 1f81d64 P3 The command center decided at 13:29Z that a classifier of the commands a model typed cannot converge, so G13 is now structural (harness/isolation.py): 1. Every listed answer source is hidden from each trial's mount namespace. The launcher (every CLI and SDK cell, the prompted and gate-0 cells) and block.py (CL7b, through the provider's codex_path_override, which points at a per-attempt wrapper) run the client's process tree under bubblewrap 0.11.1 (/usr/bin/bwrap, unprivileged). Empty tmpfs: the coordination state (run roots with oracle runs, other trials' drafts and grades; the suite cards; the v1 captures) and the fixture cache (oracles.json). Empty tmpfs with the trial's own entries bound back: ~/.cache/ws (own fixture), ~/.cache/wsr (own prompt, settings, clone, -o file, the run's neutral binaries) and ~/.claude/projects (projects outside the experiment, and the own one). A fresh per-trial folder over the shared Codex sessions folder (the clone's alias), Claude's file history, context-mode's per-project stores and Claude's /tmp area; the trial's rollouts are published to the native sessions folder after it exits. The native home and client configuration stay visible and writable, the system runtime is read-only, the network and environment are unchanged; a failed publish is recorded (finish_error), never the trial's loss. Options pass through --args (a memfd), so the namespace's PID 1 shows none; no --new-session, so the launcher's TERM reaches the client (it signals the tree below the wrapper, then KILLs the group after 30 s). systemd's InaccessiblePaths= fallback (systemd 259) is not needed: bwrap hosts every launch path. 2. G13 checks what was hidden (isolation.check): the launched row's receipt (wrapper argv, mount operations, hidden list with path and content sha256s) covers every listed location, nothing is bound back into a hidden root but the trial's own entries, the client tree ran in its own mount namespace (--info-fd), no sampled process was in the host's (nested sandboxes are counted apart), and stage 1's wrapper-only self-test passed (isolation-selftest.json; prepare refuses on a failure, gate 0 requires it and each stage-2 trial's receipt check). A trial is valid only if its own check passes. 3. The optional fanotify audit stays documented: a pid filter needs CAP_SYS_ADMIN (fanotify_init(2)), and the harness runs no sudo. 4. The command classifier (grade.reach) is a diagnostic tag only, reported under G13's classifier_diagnostic. The GPT micro-check of 1f81d64's F1 and F4 P2s concern it and are recorded as superseded. The P3 of that micro-check: the deadline and completion decisions (the hold, result-before-T, the censoring reason, the grader's effective_exit and its no-result list) read the unrounded offsets (duration_exact_s, time_to_result_exact_s, common.decision_times); the one-decimal fields are presentation only. Checked offline, no model call: harness/test_isolation.py, 26 of 26 (on the harness at 1f81d64, 3 of its 20 runnable tests fail and 16 error); the wrapper-only self-test, 43 probes hidden with ENOENT and 7 client start checks (an S7 snapshot before and after it is equal); a symtable undefined-name scan and imports; and a read-only re-grade of smoke-20261006c: every gate matches the 10:48Z grade, G13 failing for want of receipts (that run predates the wrapper), 0 of 16 trials valid. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
force-pushed
the
claude/ns2604-organic-harness-20261005
branch
2 times, most recently
from
October 6, 2026 15:45
2044b2a to
1f34363
Compare
…e, normal tier, bind triples, public receipts CC item task-ns2604-coop-20261006T143846Z: - private /tmp, /var/tmp and /dev/shm per trial (bwrap tmpfs; Claude's /tmp area bound into the private /tmp); - a per-trial ai-memory marker (workspace organic-e2e, project <trial_id>) bound above the fixture in the trial's namespace only; the grader checks every ai-memory call's returned scope: a page of another trial's scope, or a page an implicit-scope call returned (unverifiable), invalidates the trial; global queries are tagged; - host-service tags (user systemd, Docker, WSL interop, ai-memory HTTP, gateway management routes, other loopback HTTP), diagnostic only; - published receipts carry only the count and sha256 of the kept project names; - the gateway's response cache is read from the allowlisted GET /api/cache only; gate 0 passes only with the semantic cache off; G13 fails a Codex trial with a semantic-cache hit or a call log marked cacheSource "semantic". CC item task-ns2604-coop-20261006T144256Z: service_tier=default on every Codex cell (CLI line, SDK runner, CL7b provider config), checked at stage 1, at launch and in G11, which also fails on a forwarded "priority" tier. CC item task-ns2604-coop-20261006T151719Z (C1), GPT read of 2044b2a: - P2-1: G13 validates every bind as an exact (operation, source, destination) triple of the plan, with the private folder derived from the trial id; no bind may re-expose an earlier cover; the receipt's options must match its operations; - P3: the public receipt hashes every home-derived name; CL7b's options file and the self-test report are 0600; - residual channels: --unshare-ipc; the gateway's call-log payload storage and reachability without credentials are recorded (counts and HTTP status only) for the command center; - verification gap: CL7b's process tree gets the launcher's census (isolation.AppServerCensus). AMENDMENT-v1.1-20261006.md records the rulings, the gateway findings the command center must decide, and the verification. No pilot, smoke or live trial ran. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…om a trial (private runtime folder, no WSL interop) The GPT read of 2044b2a, P2-2, upheld by CC item task-ns2604-coop-20261006T151719Z (C1), in a commit of its own: - each trial gets an empty private runtime folder (bwrap --perms 0700 --tmpfs /run/user/$UID) and an empty /run/WSL, so the user manager's and session bus's sockets (systemd-run --user), rootless Docker's socket, the ssh-agent and gpg-agent sockets, and WSL interop (a Windows program would read the distribution from outside) are absent in its namespace; nothing is passed through (RUNTIME_PASS_THROUGH records the evidence); - G13 lists the runtime folder and /run/WSL, so a receipt without their tmpfs, or one binding the host's runtime folder in, fails; - negative tests (test_isolation.HostBrokers) attempt each connection inside a trial's namespace and expect failure; the stage-1 self-test probes the same (host_broker_probes), failing closed without contacting a reachable broker; - the systemd and Docker tags stay as diagnostics. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
force-pushed
the
claude/ns2604-organic-harness-20261005
branch
from
October 6, 2026 15:49
1f34363 to
a513616
Compare
4 of 6 tasks
… per trial, the GPT read of a513616 CC item task-ns2604-coop-20261006T155742Z, section 2, approved by item task-ns2604-coop-20261006T164313Z: - (A) Gate 0 amendment: the semantic cache stays on. The source evidence (OmniRoute v3.8.51 caches only explicit temperature-0 requests; codex 0.160.x sends no temperature) is pinned to the builds it was read at, and every trial records GET /api/cache before and after it: a hit, an entry change, or a missing reading voids the trial (common.gateway_cache_window, grade.gateway_cache_gate); - (B) each trial runs with --unshare-net; its only routes out are explicit forwards, each with its reason (isolation.NET_FORWARDS): the gateway's /v1 only (netfilter.py, an HTTP path filter), OTLP exports, ai-memory's /mcp and scope-checked hook routes, and Claude's model egress to api.anthropic.com and platform.claude.com. G13 requires the plan's network record, a namespace of the trial's own and the forwarder's record. The GPT read of a513616 (CHANGES_REQUESTED), folded in: - P1: call-log details only for ids the trial's own responses carried (X-OmniRoute-Request-Id), on the coordinator side, keeping only grading fields (common.gateway_calls_for_trial); - P2: gate 0 and G13 require the self-test's verified denials (grade.closure_evidence); - P2: G11 fails closed on missing or unrecognized forwarded-tier evidence (grade.tier_evidence); - P2: ai-memory pages detected by each tool's response contract, envelopes normalized apart (grade.pages_returned). Tests: 76 of 76 pass, client starts included (network negative tests, filter rules, cache-rule and tier negatives, ai-memory contracts in both clients, the collector's own-id rule); the stage-1 self-test passes with 18 of 18 network expectations. No pilot, smoke or live trial ran. Co-Authored-By: Claude Opus 5.5 <[email protected]>
… overlay, closed stores (separate, droppable) CC item task-ns2604-coop-20261006T164313Z, section 3 (c): - the home folder is a temporary overlay (bwrap --overlay-src ~ --tmp-overlay ~): nothing a trial writes there outlives it, except through explicit binds (its own fixture, clone, -o file, transcripts and private stores, and Claude Code's credentials, whose OAuth refresh rotates the stored token); projects outside the experiment are bound back read-only; - empty tmpfs over each store holding other sessions' or trials' content: ai-memory's store and hook spool, agentsview's archive, codebase-memory's and jcodemunch's indexes, Serena's logs, Claude's plans, tasks and paste cache, and Codex's own folder (history, session index, logs, memories, goals, queue), whose entries a clone links to come back as temporary overlays; Claude's history is an empty private file; - G13 lists every closure; one negative test per channel (test_isolation.ChannelClosure). Tests: 87 of 87 pass; on the round-6 commit without this one, exactly its 11 closure tests are red. The stage-1 self-test passes: 79 probes ENOENT (30 in the closed stores), 7 of 7 clients, 18 of 18 network expectations. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
force-pushed
the
claude/ns2604-organic-harness-20261005
branch
from
October 6, 2026 17:03
a513616 to
80be148
Compare
…TABLE cells, the call-id calibration cell CC item task-ns2604-coop-20261006T170607Z: - (1) forwards only for the tools under test and their local dependencies, each with its reason, path filter, tool and negative test (isolation.NET_FORWARDS): vLLM 127.0.0.1:28231 (POST /v1/embeddings, GET /v1/models) and Qdrant 127.0.0.1:21633 (GET /healthz, GET /collections, and data operations on /collections/<the trial's own QDRANT_COLLECTION_PREFIX>* only; socraticode 1.15.0 prefixes every collection and lists them before creating its own: dist/config.js:193-226, dist/services/qdrant.js:105-118, :1067, :1090-1100). [email protected] starts from the home's npm cache with npm_config_offline=true (no registry forward). Web tools and gh stay off; Claude's server-side web search is a treatment fact through the API forward. A tool under test that cannot run in a trial has its cells marked NOT-TESTABLE with the cause (isolation.tool_testability, grade.mark_testability), never NOT-READY; - (2) the pilot's first Codex trial (gate0-G1) is the designated call-id calibration cell (grade.call_id_calibration, recorded in run.json), required by gate 0 and G11, with no foreign-id fallback. Tests: 93 of 93 pass; on the round-6 head exactly the 7 round-6b tests are red. Self-test: 27 of 27 network expectations, 79 probes ENOENT, 7 of 7 clients. No pilot, smoke or live trial ran. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
force-pushed
the
claude/ns2604-organic-harness-20261005
branch
from
October 6, 2026 17:16
80be148 to
50752dd
Compare
…ode closure confirmed (GPT read of 80be148) The GPT read of 80be148 (CHANGES_REQUESTED, two P2s): - P2-1, fixed: G11 fails closed unless every required call of the trial (its gateway forward's model-call X-OmniRoute-Request-Id log, netfilter.MODEL_CALL_PATHS) has collected tier and effort evidence (grade.call_coverage, required by g11_trial_ok). The collector keeps a placeholder for a failed detail, lists the unresolved ids and tags every call with its request id; collect.py no longer filters by thread. A missing or incomplete call fails G11 and is listed by id. Regressions: A good and B detail failed, A good and B unmatched, and a required call from another thread kept. - P2-2: closed for Codex cells by the round-6b forwards (vLLM 28231, Qdrant 21633 prefix-scoped); recorded as NOT-TESTABLE for Claude cells, whose configuration points SocratiCode at 16333 and 8231 (nothing listens there). Tests: 95 of 95 pass; on the round-6b head exactly the 3 round-6c tests are red. Self-test: 79 probes ENOENT, 7 of 7 clients, 27 of 27 network expectations. No pilot, smoke or live trial ran. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
force-pushed
the
claude/ns2604-organic-harness-20261005
branch
from
October 6, 2026 17:34
50752dd to
aa3107c
Compare
…ors, own-name listing, strict call-id key, calibration first The GPT read of 50752dd (CHANGES_REQUESTED, one P1 and three P2s): - P1: only the Qdrant operations socraticode 1.15.0 performs are admitted (netfilter.judge_qdrant: collection get, create, delete, payload index, upsert, retrieve, delete, scroll and query on the trial's own prefix); grouped, batch, search, recommend, discover, snapshot and alias routes are refused; every body is parsed and every collection it names (with_lookup in both forms, lookup_from at any depth) must carry the trial's prefix (qdrant_selectors_ok). New required probes: a foreign body selector and grouped queries refused. - P2: GET /collections answers with the trial's own names only, envelope kept, framing recomputed (filter_collection_list); a required probe (0 foreign names inside, 5 outside) and a two-trial test. - P2: the call-id key is strict equality with the call log's id, with no correlationId substitution; negative case. - P2: pilot.py runs codex-native-gate0 first, collects it and runs `grade.py calibration` (exit non-zero unless it passed, calibration.json 0600) before any other cell; a failure stops the pilot. SocratiCode's NOT-TESTABLE cause for Claude cells now states the probed host ports. Tests: 100 of 100 pass; on the round-6c head exactly the 6 round-6d tests are red. Self-test: 30 of 30 network expectations, 79 probes ENOENT, 7 of 7 clients. No pilot, smoke or live trial ran. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
force-pushed
the
claude/ns2604-organic-harness-20261005
branch
from
October 6, 2026 17:52
aa3107c to
b2d44f7
Compare
…tion first on the retry path (GPT read of b2d44f7) P2, a model call without a request id disappeared from G11's coverage. netfilter.py now writes an admission row for every admitted HTTP request, before the upstream is contacted, and each outcome row names the same connection. summarize() counts the gateway forward's model calls from those rows (gateway_model_requests), keeps the request ids, and lists the calls whose response carried no id (gateway_model_calls_without_id: connection, time, method, path, outcome, status). A call with no outcome row, an unreachable gateway and a failed relay are calls without an id; a missing log gives null counts. The collector carries the count and the list into the trial's gateway record. grade.call_coverage and grade.call_id_calibration fail closed unless the admitted calls are counted, the count equals the request ids, no call lacks an id, no access-log row was unreadable and no id is shared; the calls are listed in G11's call_coverage and in calibration.json. P2, --rerun-gate0-failures could advance before the calibration passed. pilot.calibrate_first is now the one sequence for stage 2's initial path and its retry path: run the calibration cell's pending work (the cell, or the failed G1 test), collect, run grade.py calibration, and stop with exit 2 unless it passed. On the retry path a failed G1 test runs first, the check runs even when G1 passed, and the other retries follow in the stage's cell order. A passing record is never read again; a record from an earlier run that does not pass gets one fresh read. Tests (test_isolation.py, 108 with the client-start test skipped): netfilter's own Filter in front of a stand-in gateway (A with its id, B without, a call in flight, a gateway that is down), then collection and grading; and PilotStage2, which drives pilot.main with a recording step and no process. On b2d44f7's harness 10 of the 39 tests in the touched classes are red (8 fail, 2 error). The client starts (isolation.py selftest --clients) were not run for this commit: no Codex process may run on the host during the co-op's Codex window. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…g is read and filtered when the client asks for compression (separate commit) Found while measuring the compression gap the GPT read of b2d44f7 left to validate. On this host Qdrant 1.19.1 answers GET /collections with Content-Encoding gzip (or br) when asked, and Node v24.21.0's fetch, which @qdrant/js-client-rest 1.18.0 uses, asks by default. Round 6d's listing filter could not read a compressed answer and replied 502: no foreign name passed, but SocratiCode's getCollections would have failed in every Codex cell while the self-test's probe, which sent no Accept-Encoding, reported the tool testable. netfilter.py: a request whose answer the filter rewrites goes upstream without Accept-Encoding, and an answer that arrives compressed anyway is refused with 502, never passed through unread. Other routes are relayed as before. isolation.qdrant_listing_probe now asks as the client does, so the required listing expectation fails unless the listing was read and filtered. Tests: netfilter's own Filter in front of a stand-in Qdrant that compresses on request (the listing is filtered, another route keeps the header, a listing compressed anyway gets 502), and the two-trial test asks with Accept-Encoding against this host's Qdrant. Module: 109 run, 1 skipped (client starts, not run during the co-op's Codex window). On b2d44f7's harness both new checks are red (502). Co-Authored-By: Claude Opus 5.5 <[email protected]>
…vidence.json Re-registers the branch's 20 new files under evidence/artifacts/organic-e2e-20261005/ (the protocol, the pilot spec, the 2026-10-06 amendment notes and the harness, with isolation.py, netfilter.py and the focused test module) on main's manifest (c1300c1) with scripts/host_receipts.py register_file, as the docs/lanes.md hot-file protocol requires. component_matrix.py --check and new_host_grand_list.py --check pass, so neither generated report is rewritten. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
force-pushed
the
claude/ns2604-organic-harness-20261005
branch
from
October 6, 2026 18:59
b2d44f7 to
2fb12a9
Compare
Collaborator
Author
|
Closed with a record, by the command center's ruling of 2026-10-07 on the open pull requests that had no recorded owner. Not merged; the branch What it holds: the organic-invocation E2E v1.1 protocol, its pilot specification and amendment, and the promptfoo-driven harness with its run of record (21 files). The protocol text stays readable on the branch as history. Superseded by: State at closing: the last cross-family read at this head asked for changes, and two findings remain open: A supported Responses alias is admitted but omitted from model-call coverage; Bulk collection overwrites accepted calibration evidence. No fix round is planned. Reopen trigger: a dated decision that reinstates a paired organic-invocation protocol for the foundation. Reopen with |
seathatflowsinourveins
deleted the
claude/ns2604-organic-harness-20261005
branch
October 8, 2026 17:05
seathatflowsinourveins
added a commit
that referenced
this pull request
Oct 11, 2026
Codex's GitHub review of de4eed8 (4 P1, 3 P2), in both copies of the job: - Correction: GitHub stores descriptions over 65,536 characters (REST update probes up to 1,048,577 stored whole; #786 has 75,726). The bound is the gate's own, 262,144 code points. - The four decoders compose: every order of every subset, each at most once along a chain (65 forms). - validate.py and the gate take gitleaks's private-key header form, so PKCS #8 ENCRYPTED and OpenPGP key blocks are caught. - The gate does not read the graph-id form inside an http(s) URL; the graph-id name runs at most 256 characters to -code- in both scans (a quadratic 21-93 s run is now 0.05-0.1 s). - macos-user-path takes any number of leading slashes. - A test fails when validate.py gains a rule the gate does not carry. - Publication order is answered in the record: the gate is the backstop; prevention is scanning before gh pr create or edit. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Oct 11, 2026
Codex's GitHub review of de4eed8 (4 P1, 3 P2), in both copies of the job: - Correction: GitHub stores descriptions over 65,536 characters (its REST update keeps up to 262,144 UTF-8 bytes; #786 has 75,726). The bound is the gate's own, 262,144 code points. - The four decoders compose: every order of every subset, each at most once along a chain (65 forms). - validate.py and the gate take gitleaks's private-key header form, so PKCS #8 ENCRYPTED and OpenPGP key blocks are caught. - The gate does not read the graph-id form inside an http(s) URL; the graph-id name runs at most 256 characters to -code- in both scans (a quadratic 21-93 s run is now 0.05-0.1 s). - macos-user-path takes any number of leading slashes. - A test fails when validate.py gains a rule the gate does not carry. - Publication order is answered in the record: the gate is the backstop; prevention is scanning before gh pr create or edit. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Oct 11, 2026
Codex's GitHub review of de4eed8 (4 P1, 3 P2), in both copies of the job: - Correction: GitHub stores descriptions over 65,536 characters (its REST update keeps up to 262,144 UTF-8 bytes; #786 has 75,726). The bound is the gate's own, 262,144 code points. - The four decoders compose: every order of every subset, each at most once along a chain (65 forms). - validate.py and the gate take gitleaks's private-key header form, so PKCS #8 ENCRYPTED and OpenPGP key blocks are caught. - The gate does not read the graph-id form inside an http(s) URL; the graph-id name runs at most 256 characters to -code- in both scans (a quadratic 21-93 s run is now 0.05-0.1 s). - macos-user-path takes any number of leading slashes. - A test fails when validate.py gains a rule the gate does not carry. - Publication order is answered in the record: the gate is the backstop; prevention is scanning before gh pr create or edit. Co-Authored-By: Claude Opus 5.5 <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Oct 12, 2026
Codex's GitHub review of de4eed8 (4 P1, 3 P2), in both copies of the job: - Correction: GitHub stores descriptions over 65,536 characters (its REST update keeps up to 262,144 UTF-8 bytes; #786 has 75,726). The bound is the gate's own, 262,144 code points. - The four decoders compose: every order of every subset, each at most once along a chain (65 forms). - validate.py and the gate take gitleaks's private-key header form, so PKCS #8 ENCRYPTED and OpenPGP key blocks are caught. - The gate does not read the graph-id form inside an http(s) URL; the graph-id name runs at most 256 characters to -code- in both scans (a quadratic 21-93 s run is now 0.05-0.1 s). - macos-user-path takes any number of leading slashes. - A test fails when validate.py gains a rule the gate does not carry. - Publication order is answered in the record: the gate is the backstop; prevention is scanning before gh pr create or edit. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
evidence/artifacts/organic-e2e-20261005/.AMENDMENT-v1.1-20261006.md.harness/, withisolation.py,netfilter.pyand the focused test module), changed to follow those decisions.manifests/evidence.json.c1300c15b42f1a6643f3dc4172f40c01d99abdcd(origin/main after fix: relock production dependencies and preserve captured evidence #765).9e955327and rebased without conflict.git range-diff 9e955327..9835f66c c1300c15..31ff1227shows all 27 patches identical (=).2da785c2;4d490a86;e55864b5, the fixes for the GPT read of 5aa2bfd;b8c0699c, round 3 (the GPT micro-check of 87f9f1d and the CC's 12:30Z rulings, with two review passes folded in; it replaces8594ec2b,1c04fc79andd0e5350a);10aa02c0, round 4: the structural G13 (CC itemtask-ns2604-coop-20261006T132948Z) and the P3 of the GPT micro-check of 1f81d64;5a9a4cdb, round 5: the CC's 14:38Z and 14:42Z items and the accepted parts of the GPT read of 2044b2a (CC itemtask-ns2604-coop-20261006T151719Z, C1);8f09f230, round 5's P2-2 (host execution brokers), upheld at 15:17Z and kept as a commit of its own;4a34ffba, round 6: the Gate 0 cache amendment and a network namespace per trial (CC itemstask-ns2604-coop-20261006T155742Zand...164313Z), with the GPT read of a513616 folded in;4bd4b725, round 6's answer-channel closure, a separate commit the CC can drop;bdd8dea9, round 6b: forwards for the tools under test, NOT-TESTABLE cells and the call-id calibration cell (CC itemtask-ns2604-coop-20261006T170607Z);c00d39d4, round 6c: the GPT read of 80be148 (G11 requires every required call; SocratiCode's closure confirmed);72b98b09, round 6d: the GPT read of 50752dd (the Qdrant operation allowlist and body selectors, the own-name listing, the strict call-id key, the calibration cell first);1416d046, round 6e: the GPT read of b2d44f7 (every admitted model call counted, so one without a request id fails G11 and the calibration; the calibration first on the retry path too);7ade6c2f, round 6e's listing-compression fix, a separate commit the CC can drop;2fb12a99, last (docs/lanes.md hot-file protocol). It replacesb2d44f73.0d5e6506. Its1fa86506,9f3af37d,b0df1fb0and0d5e6506also changemanifests/evidence.json, so before merge the registry commit needs the same re-take on the new main.lane:foundationevidence/artifacts/organic-e2e-20261005/**: 20 new files;manifests/evidence.json: 20 newfiles[]entries, in the last commit only.SOTA sources
Each source is cited as PROTOCOL-v1.1.md §3 and §7 cite it. Where the run of record's
run.jsonrecorded a different binary, both are given.0.123.1(34f74d34e140b5e17d23770dfb2340057b1936b8), themanifests/stack.jsonpin.promptfoo evalper cell, with--repeat,-jand--no-cache.exec:providerScriptCompletionProvider(installeddist/src/providers-BUaNtf-O.js:19297-19358), with retries off throughprovider.config.maxRetries: 0(shared-CzEItb8B.js:308-313, :331-340, :939-955).openai:codex-app-serverprovider (codex-app-server-B9-ncut6.js).claude --helpon 2.1.289:--session-id,--include-hook-events,--settings,--setting-sources.v2.1.291).tool_use/tool_resultpair (§7). Subagent and Workflow calls come from the child transcripts, because on 2.1.290 and 2.1.291 those turns never carriedparent_tool_use_idin the stream.rate_limit_eventstatus (allowed,allowed_warning,rejected) and its windows'resetsAt;stop_reasonis unset on 2.1.291 stream-json assistant events).rust-v0.160.0(themanifests/stack.jsonpin; sources as cited in v1).@openai/codex-sdk0.160.0 (index.d.ts:219-240, :247-258), and the per-trial rules are checked withcodex execpolicy check.session_metaid = Lokiconversation.id.world_stateskills catalog is the session's own skill listing for decision 2.codex-cli 0.160.1(tagrust-v0.160.1), with SDK 0.160.0.v0.2.163(types.pyoptions;subprocess_cli.py:655-778).v3.7.8) behind opentelemetry-collector-contrib 0.161.0 (https://github.com/open-telemetry/opentelemetry-collector-contrib,v0.161.0).observability/collector/collector.yaml. The protocol cites lines:97-111, :120, :166, :175-177, :183, :189-192, :199-210, :233, :238-244, :263.session.id=ecosystem.task.id= trial_id; Codex Lokienv= trial_id, set through-c otel.environment(collector.yaml:199-203).tools/skill-usage/skill_usage.py --lanes --codex-root --since --until --json --call-ledger(tools/skill-usage/README.md:152-208). The shell-text rules come fromexecuted_text(:202-208).v0.43.0. The harness runsexport sessions --agent --active-since --include-automated --include-one-shot --include-children --jsonandsession tool-calls <id> --json(§7).GET /api/usage/call-logsand/api/usage/call-logs/<id>, as the protocol cites them (docs/harness-defaults.md:180). Decision 8 keeps onlypipelinePayloads.providerRequesteffort fields from them.c221cc41bc911ad0df3eaeda97acf6a1bfe9bf93,skills/mineru/SKILL.md(docs/decisions/2026-10-04-2604-e2e-fix-wave-g3-code-docs.md).49f948faa9258a0c61caceaf225e179651397431(adoption/skills/manifest.json). They are not from gh's installer.v0.11.1, installed as Ubuntububblewrap 0.11.1-1ubuntu0.3(/usr/bin/bwrap, unprivileged).bwrap --helpdescribes them:--args,--ro-bind,--bind,--bind-try,--dev-bind,--tmpfs,--perms(round 5),--proc,--unshare-pid,--unshare-ipc(round 5),--die-with-parent,--chdir,--info-fd.--info-fdrecord namesipc-namespacewhen bwrap makes one (observed on 0.11.1).v2.5.2.docs/marker-file.md: one.ai-memory.tomlmarker carries the scope for the lifecycle hooks (they walk up from the cwd) and for static MCP clients (workspace and project on every project-scoped call).ai-memory hook --helphas no scope flag.hook --check-capturereports the marker without contacting the server.v3.8.51:src/lib/semanticCache.ts:isCacheableForReadandisCacheableForWriterequire an explicittemperature: 0;open-sse/handlers/chatCore/semanticCache.ts: the cache check, andcacheSource: "semantic"on a hit;src/lib/idempotencyLayer.ts: keyed by theIdempotency-KeyorX-Request-Idheader;src/lib/providers/codexFastTier.tsandopen-sse/handlers/chatCore/serviceTier.ts: the request's tier first, then the global Codex mode, then the connection default;src/app/api/usage/call-logs/route.ts: the call-log API.scripts/hooks/secret_path_guard.py,K4_GW_ROWS. These are the gateway routes an agent command may read; the harness reads onlyGET /api/cacheand the call logs.codex-rs/protocol/src/openai_models.rs,service_tier_for_request(lines 1004 to 1009), with its testservice_tier_for_request_omits_explicit_default_tier. An explicitdefaulttier is left out of the request.codex-rs/protocol/src/config_types.rs:537:SERVICE_TIER_DEFAULT_REQUEST_VALUEis"default".codex-rs/codex-api/src/common.rs,ResponsesApiRequest(lines 279 to 304): no temperature field.README.md: the reference pattern for the network-namespace closure this round proposes to the CC. It is cited, not adopted.--pipereturns its streams./usr/share/polkit-1/actions/org.freedesktop.systemd1.policy:manage-unitsisauth_admin(auth_admin_keepwhen active), so the system manager is no broker for a trial./proc/<pid>/statfield 22 is the start time the CL7b census compares.readiness-20261005/jobs/R2a-e2-replay-20261006, outside the repository) is the pattern:bwrap --unshare-all --die-with-parent --new-session --clearenv --ro-bind /usr .... The trial wrapper keeps its read-only runtime and leaves out what a client cannot run without: the network, the environment and the home.systemctl --version: 259.5-0ubuntu3.4), systemd.exec(5)InaccessiblePaths=: the item's fallback in a transient user unit. It is not used, because bwrap hosts every launch path.SKILL.md:337-358, :394;run_eval.py:191-192, :232-234);README.md: the pattern the network namespace follows, adopted as glue only where it lacks a path filter.--unshare-net,--setenv,--overlay-srcand--tmp-overlay(installedbwrap --help; the temporary overlay of the home folder was observed to work unprivileged on this host).src/shared/constants/headers.ts(X-OmniRoute-Request-Id,X-OmniRoute-Cache,X-OmniRoute-Cache-Hit). The running composition's affinity patch (evidence/artifacts/omniroute-wire-effort-20261005/patches/045aa81f3.patch) and PR 15167's file list touch no cache code.crates/ai-memory-hooks/src/router.rs:605-611(POST/hook, POST/hook/batch, GET/handoff),crates/ai-memory-cli/src/commands/hook_spool.rs(the batch body),crates/ai-memory-mcp/src/server.rs(the tool names whose response contracts the grader reads).BASE_API_URL(https://api.anthropic.com) andTOKEN_URL(https://platform.claude.com/v1/oauth/token), and itsHTTPS_PROXYandNO_PROXYsupport.rust-v0.160.1codex-rs/codex-api/src/common.rs:279-304, identical to 0.160.0 (no temperature field).v1.15.0, commit0f5a8658):dist/constants.js:59-66anddist/config.js:193-226(QDRANT_COLLECTION_PREFIX on every collection);dist/services/qdrant.js:1067(the metadata collection is prefixed too);:105-118and:1090-1100(the collections are listed before one is created);:93,:99(no version check).npx -y [email protected] --versionprints 1.10.1 withnpm_config_offline=true.Evidence-class table
1416d046counts every admitted model call and puts the calibration first on the retry path;7ade6c2fkeeps the collection listing readable when the client asks for compressiontest_isolationwith client starts skipped: 109 run, 1 skipped, exit 0 (108 run at1416d046). Onb2d44f73's harness, 12 of the 46 tests in the five touched classes are red (9 fail, 3 error). The read's counterexample was reproduced onb2d44f73first (coverage ok, G11 true, calibration passed).isolation.py selftest --clientswas NOT re-run at2fb12a99: no Codex process may run during the co-op's Codex window.4a34ffbaimplements round 6 (A) and (B) and the GPT read of a513616test_isolation: 76 of 76 pass at4a34ffba. On the round-5 head a513616: 4 fail and 16 error (the network class cannot set up). Self-test: 18 of 18 network expectations over 10 sampled local listeners, 49 probes ENOENT, 7 of 7 clients. Read-only re-grade of smoke-20261006c: run root unchanged, no gate's pass changed against its 10:48Z grade.bdd8dea9adds the tools-under-test forwards, NOT-TESTABLE and the calibration cell4bd4b725exactly its 7 tests are red (1 fails, 6 error). Self-test: 27 of 27 network expectations, including another trial's Qdrant collection, unprefixed collections and other vLLM routes refused, and the trial's own collections reached. 79 probes ENOENT, 7 of 7 clients.4bd4b725closes the answer channels4a34ffbaexactly its 11 closure tests are red (9 fail, 2 error). Self-test: 79 probes ENOENT (30 in the closed stores), Claude's history 0 bytes inside, home writes kept in the namespace, a kept project not writable.git range-diff: 27 of 27=. The organic tree wasdcce4881before and after the rebase. The first registry's hashes equal the builder'sevidence-files-snippet.json, 16 of 16.2da785c2and4d490a86, plus 95 synthetic checks of the decision helpers, all passing. 36 of them cover the 11:43Z confirmations. They include each re-baseline key class that is never absorbed, the absorbed concurrent exit and the CL7b re-run rule. Eight checks run on smoke-20261006c's own streams and rollout, and one reads its run.json. The script was kept outside the repository and was lost when the host restarted at 13:47Z; it was not re-run in round 4.e55864b5fixes the seven P2 findings of the GPT read of 5aa2bfd536487a4and all 10 pass on this head. The script was kept outside the repository and was lost when the host restarted at 13:47Z; it was not re-run in round 4.b8c0699cfixes the four P2s of the 87f9f1d micro-check and adds the 12:30Z Codex timing87f9f1d7, 7 atfeebace4and 9 at1c04fc79(the two earlier pushes of this round), and all 12 on this head. Besides the original nine, they cover piped and sequenced listings, xargs options and multi-action find, and everyday agent commands. They include a realtimeout --kill-afterSIGKILL (rc 137), a realprintfCodex event stream through the launcher, and a read-only re-grade of smoke-20261006c in which all 11 Codex trials carry both times. The scripts were kept outside the repository and were lost when the host restarted at 13:47Z; they were not re-run in round 4.10aa02c0hides every listed answer source from each trial's mount namespace, makes G13 a check of the receipt and the classifier a diagnostic, and fixes the 1f81d64 P3harness/test_isolation.py, committed: 26 of 26 pass on this head. On the harness at1f81d645, 3 of its 20 runnable tests fail and 16 error. The stage-1 self-test: 43 probes, each a real file in a hidden location (this run's sibling trial included), return ENOENT in the namespace and read outside it. The own inputs, the native client configuration and a project outside the experiment stay readable. 7 client start checks pass:claudeandcodex --version, Codex's sandbox nested in bwrap, the CL6 and CL7 SDK imports, andcodex app-server --helpthrough the CL7b wrapper. An S7 snapshot before and after the self-test is equal.5a9a4cdbimplements the 14:38Z and 14:42Z items and the accepted P2-1 and P3 of the GPT read of 2044b2a, with--unshare-ipcand CL7b's censusharness/test_isolation.pyat5a9a4cdb: 52 of 52 pass, client starts included. The same 52 tests on 2044b2a: 11 fail, 16 error and 25 pass. Self-test: 49 probes ENOENT, 7 of 7 client checks, IPC namespaces distinct from the host's, no home path or home slug in the report.8f09f230(P2-2) makes the user manager, rootless Docker and WSL interop unreachable from a trial5a9a4cdb: exactly P2-2's 8 checks are red (7 fail, 1 error) and the other 51 pass; on 2044b2a, 17 fail and 17 error. Inside the namespace, each connection attempt fails:systemd-run --user(cannot connect to the user scope bus), Docker's/_pingoverdocker.sock(curl exit 7), andcmd.exe /c ver(interop connect failed). Outside, the user manager answereddegradedand DockerOK.GET /api/cachegavesemanticCacheEnabled: truewith 0 hits and 0 entries.GET /api/usage/call-logs?limit=20answered without credentials: 1 of the 20 rows stores request and response bodies, and 0 have pipeline details. From a trial's namespace,/api/healthand/api/usage/call-logs?limit=1answered 200 without credentials.grade.py meter-calibration --run-root <smoke-20261006c>: exit 0, read-only, p90 0.06 five-hour and 0.01 seven-day over 3 organic Claude trials.--writewas run on a scratch copy of the ledger only.ast.parseand import of all 13.pyfiles; a stdlibsymtablescan (pyflakes is not installed): 0 names;node --check sdk_codex.mjs: exit 0.grade_run()of smoke-20261006c withgrade.pysha256272baae2…; the output was written outside the run root, which it left unchanged. Every gate matches the 10:48Z grade. G13 fails under the structural rule for want of receipts, since that run predates the wrapper, and 0 of its 16 launched trials are valid. The classifier's diagnostic still tags 3 trials and finds no answer-source read.nice -n 19 python3 scripts/validate.py: exit 0,hashed_files10334.git diff --check origin/main...HEAD: exit 0.~/.local/state/native-agent-stack/coordination/ns2604-coop/organic-e2e-20261005/v11/smoke-20261006c/(private, outside the repository):run.json,gate0.json,grades/grade-2026-10-06T104807Z.json.Local commands run
Deliberately not run:
2fb12a99, the client starts (isolation.py selftest --clientsand the module's client-start test): no Codex process may run on the host during the co-op's Codex window;cat, the clients'--versionand--help, and the broker probes only, with no model call.No repository test imports the harness (
git grep organic-e2e-20261005 -- tests/ scripts/: no match). The builder reported earlier, on exported copies, the same unittest failure sets for base9e955327and for37417f4fplus the manifest entries. That is the builder's report and was not rerun here.Run of record: smoke-20261006c
Run conditions:
37417f4f(now1332376f), two commits before the pre-decision harness head.@openai/codex-sdk0.160.0.grade.py4f2c07b6…. That grader is now superseded by round 5's (5a9a4cdb); the re-grade results are noted below.Passed:
Did not pass:
~/projects, and the env arms read~/code/native-agent-stackand~/.codex/RTK.md.Not yet run live:
fee6e6df);R4 labels are also still pending:
label_vector_sha256is null.Decided (CC 10:55Z)
Command-center item
task-ns2604-coop-20261006T105529Zdecided the open questions.AMENDMENT-v1.1-20261006.mdrecords each decision with its number, and2da785c2implements them. PROTOCOL-v1.1.md stays verbatim.no_result_diagnosisand holds its cell (HOLD.<cell>) until the operator has diagnosed it.not_exposed_path_only).rate_limitedand re-run.-jtrials of the Codex cell in flight (3 for codex-native and codex-env in the pilot), plus a CL7b trial that ran across it.hooks_state, the project tables, trusted sets, trust fields, removed projects), ~/.claude/settings.json (the probe's hash) or the arm-defining files. Those, any second change, and a change seen only at a block's end still stop the run.Confirmed (CC 11:43Z), by item
task-ns2604-coop-20261006T114319Zand implemented in4d490a86:holds_cell()depends only on the client and the deadline evidence (the elapsed time and when the result arrived), never on the cell, arm, kind or stage.prepare.py --answer-source-path) and earlier graded smoke or pilot receipts.rebaseline_cost_ok()enforces it at every re-run, with one trial per arm as the fallback, and the run stops past it.pilot.pyrecalibrates it to the measured per-window p90 after the first stage-4 Claude block with usable readings, writingmeter-calibration.json.grade.py meter-calibration.Not in this PR:
ackledger row is the co-op's to append.Verification still due:
5aa2bfdc(seven P2s) and the micro-check of87f9f1d7(four P2s) are fixed ine55864b5andb8c0699c(see below). Round 6's commits,4a34ffba,4bd4b725,bdd8dea9,c00d39d4and72b98b09(on8f09f230), are the new delta for the co-op's GPT read, before the command center's ACK.b83c8794..70d46df6(were6e912b0d..16ff012b) were checked through live smokes and self-tests, not reread line by line.~/.cachefor about 80 fixtures of about 228 MB each is unchecked.Round 6: the Gate 0 cache amendment, a network namespace per trial, the channel closure (CC 15:57Z and 16:43Z)
AMENDMENT-v1.1-20261006.md("Round 6") records each item in full.1. The Gate 0 amendment (CC item 155742Z (A), dated 2026-10-06):
/api/cachebefore and after it. A hit at either reading, an entry change, or a missing or failed reading voids it.2. A network namespace per trial with explicit forwards only (item 155742Z (B)):
/v1/...only/v1/logs,/v1/metrics,/v1/traces/mcp; the hook routes, which must name the trial's own scopeapi.anthropic.com:443andplatform.claude.com:443, throughHTTPS_PROXY/api/usage/call-logsand/api/healthrefused by the filter (the gateway never sees them), sampled listeners refusing the connection, no gateway port for Claude, and/v1still answering.gh, npm and socraticode's backends are not forwarded. Any further forward goes intoNET_FORWARDSwith its reason.3. The GPT read of a513616 (CHANGES_REQUESTED), folded in:
X-OmniRoute-Request-Id, recorded by its gateway forward), on the coordinator side, keeping only grading fields. A test checks that a foreign id is never requested. That header and the call log's id match is read from source, not yet observed; with no match, G11 stays failed.network-closure) and G13 require the self-test's 13 verified denials.default, and every call's forwarded tierdefaultor none. Missing or unrecognized evidence fails, with the call ids.Round 6e (
1416d046, and7ade6c2fas a separate commit; the GPT read of b2d44f7, two P2s):isolation.py selftest --clientsand the module's client-start test startcodex, and no Codex process may run on the host during the co-op's Codex window. Their last run was atb2d44f73(7 of 7 clients). Everything below ran with client starts skipped.admittedrow for every admitted HTTP request, before the upstream is contacted. Each outcome row names the same connection.Filterruns behind a stand-in gateway (A with its id, B without). The result then goes throughgateway_calls_for_trial,call_coverage,g11_trial_ok, andcall_id_calibrationon the written record. No test exercisescollect.py'smainpassing the forward record, orgrade_runreading the exit row.b2d44f73first: 2 admitted calls, 1 id, coverage ok, G11 true, calibration passed.pilot.calibrate_firstis now the one sequence for stage 2's initial path and its--rerun-gate0-failurespath: run the calibration's pending work, collect, rungrade.py calibration, and stop with exit 2 unless it passed.pilot.mainitself with a recording step, with G1 and a prompted check both failed among the cases. No process starts.7ade6c2f, droppable). Measured on this host: Qdrant 1.19.1 compresses the listing on request, and Node's fetch asks by default. Round 6d's filter then answered 502, so SocratiCode's listing would have failed in Codex cells while the self-test's probe passed.Accept-Encoding, and a listing compressed anyway gets 502./v1/responses,/v1/chat/completionsor/v1/completions. Any other admitted/v1request is not required to carry an id.b2d44f73's harness, 12 of the 46 tests in the five touched classes are red (9 fail, 3 error).Round 6d (
72b98b09, the GPT read of 50752dd, one P1 and three P2s):with_lookupin both forms,lookup_fromat any depth) must carry the trial's prefix./collectionsanswers with the trial's own names only. The envelope is kept and the framing recomputed. A required probe shows 0 foreign names inside against 5 outside, and a two-trial test confirms the same.id, with no correlation-id substitution, and a negative case for it.pilot.pyrunscodex-native-gate0first, then collects it and runsgrade.py calibrationbefore any other cell. A failure stops the pilot.c00d39d4exactly the 6 new tests are red; there, the two-trial test shows trial B seeing trial A's collection. The self-test meets 30 of 30 network expectations.Round 6c (
c00d39d4, the GPT read of 80be148, two P2s):collect.pyno longer drops calls by thread.bdd8dea9, exactly the 3 new tests are red.4. Round 6b (
bdd8dea9, CC item 170607Z):Forwards. The only new forwards are for the tools under test and their local dependencies:
/v1/embeddingsand GET/v1/models;/healthz, GET/collections(names only; SocratiCode cannot create its collections without it), and data operations on/collections/<the trial's own QDRANT_COLLECTION_PREFIX>*only.Each forward records its reason, filter and tool, and has negative tests.
chrome-devtools starts from the home's npm cache, offline; there is no npm forward.
Web tools and
ghstay off. Claude's server-side web search is a treatment fact, reached through the API forward.NOT-TESTABLE. A tool that cannot run in a trial gets its cells marked NOT-TESTABLE with the cause, never NOT-READY. On this host, SocratiCode in Claude cells is NOT-TESTABLE: Claude's configuration points it at ports where nothing listens.
The calibration cell. The pilot's first Codex trial (
gate0-G1) calibrates the call-id key. Gate 0 and G11 require it, with no foreign-id fallback.5. The answer-channel closure (
4bd4b725, droppable):Round 5: residual channels, the normal tier and the GPT read of 2044b2a (CC 14:38Z, 14:42Z and 15:17Z)
AMENDMENT-v1.1-20261006.md("Round 5") records each item in full. Two gateway items need the command center first.1. The gateway's response cache is not confirmed off (14:38Z item):
semanticCacheEnabled: true, with 0 hits and 0 entries.temperature: 0, and codex-cli 0.160.0's Responses request has no temperature field.gateway-cachecheck passes only while the cache is off. G13 fails a trial whose hit count rose or whose call logs showcacheSource: "semantic".2. The gateway's logs and management routes, returned to the CC (15:17Z item):
/api/healthand/api/usage/call-logs?limit=1both answered 200).bwrap --unshare-net) whose only routes out are explicit forwards for what the treatment needs: the gateway's inference path, the OTLP collector, the ai-memory server and the URL-configured MCP servers. The pattern is anthropics/sandbox-runtime v0.0.78;/v1and/apishare one port. Either the forward admits/v1/...only, or OmniRoute's management routes require authentication during trial runs.execute_shell_command), hcom, the codebase-memory daemon, agentsview, Loki and Grafana, among 110 TCP ports listening at 15:47Z.gateway-management-api,local-service-http) as diagnostics./api/settingsand/api/settings/cache-configthrough script files. The command guard does not allowlist either route, and both return decrypted secrets. Only cache keys and key names were printed, and nothing was stored. The harness reads only the allowlisted routes.3. The normal service tier (14:42Z item):
service_tier=defaultexplicitly: CL3, CL4 and the prompted cells on the line, CL7 in the runner'sconfigOverrides, and CL7b in the provider'scli_config.defaultout of the request, so OmniRoute's own Codex tier settings decide what goes out. G11 therefore also fails on a forwardedprioritytier (pipeline details, which are on only for pilot runs).codexServiceTiervalue is in/api/settings, which agents may not read. The operator can read it in a terminal.4. Residual channels (14:38Z item, extended by the GPT read):
/tmp,/var/tmpand/dev/shmare private per trial. Each trial also gets its own IPC namespace (--unshare-ipc), and G13 fails a trial without one.organic-e2eand project<trial_id>is bound above the fixture in the namespace only.5. The GPT read of 2044b2a (15:17Z item, C1):
work/lastbound onto the trial's own fails, as do a private store bound from the shared store and a receipt naming another private folder.test_isolation.G13Check.-home-<user>-...slugs) wherever it sits. The exact mount operations stay private: the ledger is 0600, and CL7b's options file and the self-test report are 0600.8f09f230.--perms 0700 --tmpfs /run/user/$UID) and an empty/run/WSL.test_isolation.HostBrokers) attemptsystemd-run --user, Docker's/_pingoverdocker.sockandcmd.exeinside a trial's namespace, and expect failure. The stage-1 self-test probes the same.The wrapper's argv in round 5, for a Codex trial. As before, a process listing shows only
/usr/bin/bwrap --args <fd> -- bash -lc <line>. This is the public form: the 68 projects outside the experiment become a count and a hash.Without
8f09f230, the runtime folder is bound in (--bind-try /run/user/1000 /run/user/1000) and/run/WSLstays visible.Residual channels after round 5:
_globalscope: a page another trial wrote there is not caught; a write to it is tagged.Round 4: the structural G13 (CC 13:29Z) and the 1f81d64 P3
Command-center item
task-ns2604-coop-20261006T132948Zdecided that a classifier of the commands a model typed cannot converge, because shell grammar always has another form. G13 is now structural, inharness/isolation.py(10aa02c0):codex_path_override, which points at a per-attempt wrapper) run each client's process tree under bwrap.~/.cache/ws(its own fixture),~/.cache/wsr(its own prompt, settings, clone and -o file, and the run's neutral binaries), and~/.claude/projects(the projects outside the experiment, and its own)./tmparea. The trial's rollouts are published to the native sessions folder after it exits; a failed publish is recorded asfinish_errorand never costs the trial's exit row.--info-fd);Gate 0 requires the self-test and each stage-2 trial's check, and a trial is valid only if its own check passes.
The wrapper's argv in round 4 (superseded by round 5's, above), as bwrap parses it for a Codex trial (a process listing in the namespace shows only
/usr/bin/bwrap --args <fd> -- bash -lc <line>):A Claude trial has its settings file instead of the clone, and its own project bound back after the tmpfs over
~/.claude/projects. CL7b's wrapper execs bwrap with the same options and the real codex.Limits in round 4 (round 5 narrows them, above):
/tmpand/dev/shmstay shared outside Claude's own area.Round 3: the GPT micro-check of 87f9f1d and the CC's 12:30Z rulings
Findings 1 and 4 below tune the classifier. The GPT micro-check of 1f81d64 found them still partial, and round 4 supersedes them; findings 2 and 3 stand.
The micro-check requested changes for four P2s, all fixed in
b8c0699c:rg --files/-l/-c,git ls-files, cp/mv, Glob, the Grep tool's name and count modes, andfind -exec grep -lorxargs grep -l. find and xargs are judged by the programs they run: every find action, with xargs options read as the installed GNU findutils xargs 4.10.0--helplists them.ls <dir> | head -n 5,| sort,| grep x) reads the listing, not a file, so it is no read.git showafter global options such as-C <dir>, the Read tool, the Grep tool in content mode).block.pyrefuses such a block, andprepare.pyrefuses--repeat-overrideabove 1 whilecodex-app-serveris among the cells. That holds until each repetition gets its own attempt; a re-run is a new block with a fresh attempt.timeout_killed); a kill before T does not hold.cdin the command, and, for an operand another command produced or a heredoc program, every path the command names. A listing beside or piped into a read is therefore not scanned. Known limits: a script file (python3 x.py) and acdfrom an earlier call are not followed. Other tasks' rollouts stay tags.Confirmed (CC 12:30Z), item
task-ns2604-coop-20261006T123036Z:turn.completed).AgentMessage,task_complete).completion_timesfor every trial andtiming_recordfor the Codex ones. On smoke-20261006c all 11 Codex trials carry both times.Fixes from the GPT first-pass read of 5aa2bfd
The read requested changes for seven P2 findings. All are fixed in
e55864b5, andAMENDMENT-v1.1-20261006.mdrecords them:interruptedexit, and a launcher failure after launch getslauncher_error_after_launch; both are carried forward.rate_limited, sets STOP.codex and is carried forward. Gateway counts alone never mark a trial.The same commit also tightens the meter calibration (point 5): trials with fewer than two readings, and windows that rolled over mid-trial, are left out.
Decision record
evidence/artifacts/organic-e2e-20261005/AMENDMENT-v1.1-20261006.mdrecords the command center's dated decisions (itemstask-ns2604-coop-20261006T105529Z,...114319Z,...123036Z,...132948Z(the structural G13),...143846Z,...144256Zand...151719Z(round 5),...155742Z,...164313Zand...170607Z(round 6)). For each decision it gives the change, the points to confirm and the offline evidence. PROTOCOL-v1.1.md §13 lists the decisions as they stood before.Host evidence
Not applicable: this PR adds or changes nothing under
evidence/hosts/.Checklist
permissions: {}and grant each job only what it needs. Not applicable: no workflow changes.🤖 Generated with Claude Code
Open findings at 2fb12a9 (not fixed; the harness is shelved)
Under the evaluation rule of 2026-10-06 (the decision record in #802) this harness runs no trial as a gate, and the re-scoped S4 read-back does not use it. No further round follows. The PR stays a draft at this head.
Two P2 findings of the cross-family read at this head stay open. Fix both, with their regressions, before the harness is ever used as a runnable tool:
/v1/v1/<path>to the same routes, so/v1/v1/responsesis admitted by the forwarder but is not counted as a model call (netfilter.py: admission againstMODEL_CALL_PATHSinsummarize). Coverage and calibration can pass on the canonical call alone. Fix: refuse unsupported aliases before connecting, or canonicalize and count every admitted alias; add a good-canonical plus alias-without-id case and an alias-in-flight case.calibrate_firstkeeps a passing gateway record; the general stage-2 collection then re-fetches and overwrites it (pilot.py,collect.py), so a later list failure can fail Gate 0 on evidence that had passed. Fix: preserve validated calibration snapshots across later bulk collection, handle an empty trial selection explicitly, and add a regression with a saved passing G1, an unrelated retry and unavailable historical rows.Recorded limitations, not findings of that read: a required call is a request on the three canonical model routes only; the duplicate-key and method-override gaps of the earlier read are untouched; whether the gateway's request-id header equals the call log's id is unobserved;
calibrate_firstskips with a log entry when a restricted run has no calibration cell; no test coverscollect.py'smainpassing the gateway record.State of checks at this head: the client self-test and the isolation test module pass (109 run, none skipped);
validateis green;osv-scannerfails on an advisory in two lockfiles this PR does not touch, which #797 fixes. The registry must be redone on the then-current main before any merge.