Skip to content

Landscape-sweep OmniRoute lane: exact context-mode read rules for the pinned skills; keep the gateway key out of model shells - #393

Merged
seathatflowsinourveins merged 4 commits into
mainfrom
claude/sweep-lane-skill-roots-20260927
Sep 27, 2026
Merged

Landscape-sweep OmniRoute lane: exact context-mode read rules for the pinned skills; keep the gateway key out of model shells#393
seathatflowsinourveins merged 4 commits into
mainfrom
claude/sweep-lane-skill-roots-20260927

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Sep 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

What

The landscape-sweep OmniRoute smoke (wf_6576675c-0d6, harness at b9abcc5) passed every harness check. Both GPT-6 jobs were ok, the copy check matched 2/2, and nothing was lost or deviated. One gap remained: both GPT-6 workers reported skills_used: [].

Their events show why:

  • Each tried to load its pinned skills with context-mode's ctx_execute_file:
    • discovery: search-first, iterative-retrieval;
    • fit: verification-before-completion, supply-chain-risk-auditor, fp-check.
  • The paths were in ~/.agents/skills.
  • context-mode 1.0.169 refuses a path outside its project directory, which for this lane is the runner's <work-dir>/empty. A Read(...) allow rule in <project>/.claude/settings.json is the only way past it (build/security.js evaluateProjectContainment and readToolPermissionPatterns, issue Pin vendor skills for CC lane messaging, evals and logs #852).

What build_args.py --gpt6-provider omniroute now does:

  • It writes <work-dir>/empty/.claude/settings.json with one exact Read(...) rule per file under $HOME/.agents/skills, where install_skills.py puts the pinned skills. Codex lists user skills from there and from $CODEX_HOME/skills (codex-rs/ext/skills/src/host_roots.rs at rust-v0.157.1); in the lane, the latter holds only Codex's .system cache.
  • Symlinked directories are followed, each real directory is listed once, and a path containing * or ? is left out.
  • The rules are exact because context-mode also matches the raw path, so a <root>/** rule would admit <root>/../elsewhere and siblings (GPT-6 review r1).
  • staged.json records the symbolic root and the file count, and no host path.
  • A native restage removes the file.
  • Codex reads no .claude/ files, so no Codex project layer is added.
  • The lane config gains [shell_environment_policy.filters] excluding the gateway key variable. Codex 0.157.1 defaults ignore_default_excludes to true, so a real key would otherwise reach model-run commands.

Residual, recorded in the README. context-mode normalizes backslashes before matching, while its executor opens the literal Linux name. A file whose name holds backslashes that normalize to a listed path would pass too (GPT-6 review r2). No rule can exclude it. The #852 boundary limits ctx_execute_file only: ctx_execute, which the stack-worker profile enables, runs code with the user's own file access, outside Codex's read-only sandbox.

The README also records the 2026-09-27 web-search measurement:

  • /v1/alpha/search is 404 at a58000c7.
  • Upstream PR #13788 adds the route, backed by OmniRoute's search registry (duckduckgo-free when keyless), not OpenAI's hosted search.

The stale note that the default worker profile was still to come is removed, since #389 merged it.

Verification

  • Failing controls: the first revision fails the new tests; the wildcard rule admits the traversal under context-mode's own matcher.
  • Tests: CONTEXT_MODE_SECURITY_JS=<installed build/security.js> python3 -m unittest tests.test_saturation_ledger tests.test_landscape_sweep_harness gives 168 run, OK (3 skipped). The behavioral test runs context-mode's real matcher and skips without the variable.
  • python3 scripts/validate.py and python3 scripts/build_ecosystem.py --check: passed.
  • Live probe through the gateway, with a synthetic non-placeholder key:
    • the variable is absent in GPT-6's shell;
    • fp-check/SKILL.md reads through its exact rule;
    • a .. traversal is refused.
  • GPT-6 review: r1 and r2 as in the review comment. r2's remaining case is the recorded residual.
  • Integration: smoke 2 (wf_e4dee61f-e66) runs with the SubagentStart token-lanes hook active. The result is posted before merge.

SOTA sources

  • context-mode 1.0.169, build/security.js: evaluateProjectContainment, readToolPermissionPatterns, evaluateFilePath, fileGlobToRegex.
  • Codex rust-v0.157.1:
    • codex-rs/ext/skills/src/host_roots.rs:96-113;
    • codex-rs/config/src/shell_environment_policy.rs:136;
    • codex-rs/protocol/src/config_types.rs:222-230;
    • codex-rs/protocol/src/shell_environment.rs.

🤖 Generated with Claude Code

https://claude.ai/code/session_012qwXNtFyUhY5x2dtrm7tG5

Scout @claude
Landscape-sweep OmniRoute lane: let context-mode read Codex's skill r…
ec9f071 
…oots

The OmniRoute smoke (wf_6576675c-0d6) passed every harness check except one.
Both GPT-6 workers reported skills_used: []. Each tried to load its pinned
skills (search-first, iterative-retrieval, verification-before-completion,
supply-chain-risk-auditor, fp-check) with context-mode's ctx_execute_file.
context-mode 1.0.169 refuses paths outside its project directory, here the
runner's <work-dir>/empty, unless a Read(...) allow rule in
<project>/.claude/settings.json names them (build/security.js
evaluateProjectContainment and readToolPermissionPatterns, issue #852). The
native lane's GPT-6 had reported them used.

build_args.py now writes <work-dir>/empty/.claude/settings.json for the
gateway lane. It allows Read on Codex's user skill roots only,
$CODEX_HOME/skills and $HOME/.agents/skills
(codex-rs/ext/skills/src/host_roots.rs at rust-v0.157.1), as absolute globs,
since context-mode matches literally. staged.json records the roots
symbolically. A native restage removes the file. Codex reads no .claude/
files, so no Codex project layer is added.

The README records the rule and the 2026-09-27 web-search measurement:
- /v1/alpha/search is 404 at a58000c7;
- upstream PR #13788 answers from OmniRoute's search registry
  (duckduckgo-free when keyless), not OpenAI's hosted search.
The stale note about the default worker profile goes, since #389 put it on
main.

The failing control: the new test errors against main's build_args.py
(no settings file).
Tests: tests.test_saturation_ledger + tests.test_landscape_sweep_harness,
167 run, OK (3 skipped). validate.py and build_ecosystem.py --check passed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_012qwXNtFyUhY5x2dtrm7tG5
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 27, 2026
Scout @claude
#393 GPT-6 review repair: exact skill-file rules; filter the key from…
45a9d77 
… model shells

The GPT-6 review of ec9f071 returned needs_changes with one medium finding.
context-mode 1.0.169 matches an allow rule against the raw path as well as
the resolved one (security.js evaluateFilePath), so the <root>/** rules also
admitted <root>/../elsewhere, sibling paths under the root and outward
symlinks. The claim of "there and nowhere else" did not hold, and the test
only checked serialization.
- The lane now gets one exact Read(...) rule per file under
  $HOME/.agents/skills. That is where install_skills.py puts the pinned
  skills; the lane's $CODEX_HOME/skills holds only Codex's .system cache.
  Symlinked directories are followed, each real directory is listed once,
  and a path containing * or ? is left out. context-mode escapes and anchors
  every other character, so an exact rule matches only its path.
- staged.json records the symbolic root and the file count.
- New behavioral test: test_context_mode_reads_only_the_listed_skill_files
  runs context-mode's own readToolPermissionPatterns and
  evaluateProjectContainment on the staged file (set CONTEXT_MODE_SECURITY_JS
  to an installed build/security.js). Listed files pass; a sibling, an
  outside file and two .. traversals do not. Its control shows a <root>/**
  rule admitting the traversal. The first revision fails it (sibling and
  traversal allowed).

Also, a residual the gateway owner found while reviewing their Codex
templates: Codex 0.157.1 applies its default *KEY*/*SECRET*/*TOKEN* excludes
only when ignore_default_excludes is false, and that defaults to true
(codex-rs/config/src/shell_environment_policy.rs:136). A real gateway key
would therefore reach model-run commands. The lane config now sets
[shell_environment_policy.filters] OMNIROUTE_API_KEY = "exclude"
(codex-rs/protocol/src/config_types.rs:222-230).

Live probe on a lane staged from this revision, with a synthetic
non-placeholder key through the keyless gateway:
- GPT-6's shell reported the variable absent;
- ctx_execute_file read fp-check/SKILL.md;
- ctx_execute_file refused ~/.agents/skills/../../code/.../README.md.

Tests: 168 run, OK (3 skipped), with CONTEXT_MODE_SECURITY_JS set.
validate.py and build_ecosystem.py --check passed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_012qwXNtFyUhY5x2dtrm7tG5
@seathatflowsinourveins seathatflowsinourveins mentioned this pull request Sep 27, 2026
Merged
4 tasks done
Scout @claude
#393 GPT-6 re-check: record context-mode's backslash residual and ctx…
4f6cfd8 
…_execute's reach

The GPT-6 re-check of 45a9d77 confirmed the exact rules reject traversal,
sibling and outside paths. It admits listed files, references and symlinked
skill directories. The key filter removes the variable from model commands
while env_key still reaches the provider.

One case stays open, and it is upstream matcher behavior. context-mode
1.0.169 turns backslashes into slashes before matching (security.js
evaluateFilePath) but its executor opens the literal Linux name. A planted
symlink named <home>/.agents\skills\demo\SKILL.md therefore passed and read
an outside file. No rule can exclude it, since deny matching normalizes the
same way. Per the bounded review loop this is recorded as a residual, not
fixed:
- no such file exists;
- creating one takes write access that ctx_execute already has.
The README and comments also state the wider fact the review surfaced: the
#852 boundary limits ctx_execute_file only, while ctx_execute (left enabled
by the stack-worker profile) runs code with the user's own file access
outside Codex's read-only sandbox.

Tests: 168 run, OK (3 skipped), with CONTEXT_MODE_SECURITY_JS set.
validate.py and build_ecosystem.py --check passed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_012qwXNtFyUhY5x2dtrm7tG5
@seathatflowsinourveins

seathatflowsinourveins commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

GPT-6 cross-family review, two rounds (gpt-6-astra, effort max, read-only):

Round Head Verdict Finding Now
r1 ec9f071 needs_changes Medium: context-mode 1.0.169 also matches an allow rule against the raw path, so <root>/** admitted <root>/../outside, siblings and outward symlinks. The test checked only serialization. Fixed in 45a9d77, detailed below
r2 45a9d77 needs_changes Medium: context-mode normalizes backslashes before matching, while the executor opens the literal Linux name. A planted <home>/.agents\skills\demo\SKILL.md symlink passed and read an outside file. Recorded as a residual in 4f6cfd8, detailed below

What 45a9d77 changes:

  • The lane gets one exact Read(...) rule per file under $HOME/.agents/skills (223 on this host). It follows symlinked directories, lists each real directory once, and leaves out any path containing * or ?.
  • A new behavioral test runs context-mode's own readToolPermissionPatterns and evaluateProjectContainment against the staged file. Listed files pass; a sibling, an outside file and two .. traversals do not. Its control shows the wildcard admitting the traversal, and the first revision fails the test.
  • It also adds [shell_environment_policy.filters] excluding the gateway key variable. The gateway owner found this residual: Codex 0.157.1 defaults ignore_default_excludes to true.

Why the r2 case is a residual and not a fix: no rule can exclude it, because deny matching normalizes the same way. No such file exists, and creating one needs write access that ctx_execute already has. The README and comments now state the wider fact the review surfaced: the #852 boundary limits ctx_execute_file only, and ctx_execute (enabled by the stack-worker profile) runs code with the user's own file access, outside Codex's read-only sandbox.

r2 confirmed the rest:

  • Listed files, references and symlinked skill directories pass. Traversal, siblings, outside files, and newline or carriage-return variants fail.
  • The filters entry removes the variable from model commands while env_key still reaches the provider. It is std::env::var in model-provider-info/src/lib.rs.
  • The stack-worker profile has no conflicting shell-environment policy.
  • The metadata, cleanup and evidence hashes are correct.

Live probe on a lane staged from 45a9d77, with a synthetic non-placeholder key through the keyless gateway:

  • GPT-6's shell reported the variable absent;
  • ctx_execute_file read fp-check/SKILL.md;
  • ctx_execute_file refused ~/.agents/skills/../../code/native-agent-stack/README.md.

Tests at 4f6cfd8: 168 run, OK (3 skipped), with CONTEXT_MODE_SECURITY_JS set so the behavioral test runs. validate.py and build_ecosystem.py --check passed.

Integration: smoke 2 (wf_e4dee61f-e66, harness 45a9d77, host SubagentStart token-lanes hook active) is running. Its first attempt hit the Claude account session limit and is archived as a failed attempt. The result will be posted here before merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_012qwXNtFyUhY5x2dtrm7tG5

@seathatflowsinourveins seathatflowsinourveins changed the title Landscape-sweep OmniRoute lane: let context-mode read Codex's skill roots (GPT-6 reported no skills in the smoke) Landscape-sweep OmniRoute lane: exact context-mode read rules for the pinned skills; keep the gateway key out of model shells Sep 27, 2026
@seathatflowsinourveins

seathatflowsinourveins commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Integration: smoke 2 passes.

  • Run: wf_e4dee61f-e66, harness 45a9d77. Its code is identical to 4f6cfd8, which changes only comments and the README.
  • Gateway: OmniRoute BUILD_SHA dd6e9607e.
  • Host SubagentStart token-lanes hook: active. Both agents' transcripts carry the injected block.

Results:

  • Step-4 probe: exit 0 in 55 s, no limit; v0.5.0 and 129,658 stars, both exact.
  • usage_record.py: complete, 5 children (2 claude-opus-5-5, 3 claude-sonnet-5), no effort mismatches.
  • convert.py:

    • failure fields: lost [], degraded_discovery [], effort_deviations [], retained_failures {}, reopened_layers [];

    • GPT-6: gpt6_by_status ok 2, gpt6_copy_check match 2. The Sonnet wrappers returned the raw result verbatim with the hook active;

    • skills_usage now has GPT-6 entries:

      • discover_gpt6: search-first, iterative-retrieval, supply-chain-risk-auditor, verification-before-completion;
      • fit_gpt6: verification-before-completion, supply-chain-risk-auditor, fp-check.

      The first smoke (wf_6576675c-0d6, before this PR) had none.

    • calls: GPT-6 made 4 web searches, 8 page fetches and 40 GitHub API calls.

    • Outcome: 8 proposals, 3 survivors.

Attempts are retained in the private work directory:

  • smoke-omniroute: before this PR, skills_used: [];
  • smoke2-session-limit: lost to the Claude account session limit, recorded with its cause;
  • smoke2-omniroute-hook: this run.

🤖 Generated with Claude Code

https://claude.ai/code/session_012qwXNtFyUhY5x2dtrm7tG5

@seathatflowsinourveins
seathatflowsinourveins merged commit ec27a30 into main Sep 27, 2026
24 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/sweep-lane-skill-roots-20260927 branch September 27, 2026 12:55
@seathatflowsinourveins seathatflowsinourveins mentioned this pull request Sep 27, 2026
Merged
seathatflowsinourveins pushed a commit that referenced this pull request Sep 27, 2026
Scout @claude
Landscape sweep: the gateway lane home carries the host's Codex user …
b65ff10 
…instructions (AGENTS.md)

The OmniRoute lane (build_args.py --gpt6-provider omniroute) runs Codex with
CODEX_HOME set to a lane-local home holding only config.toml and the
stack-worker profile. Codex reads $CODEX_HOME/AGENTS.md as global
instructions, so the lane's GPT-6 got neither the repository's top rule nor
RTK's instructions, which the native lane's workers get from
~/.codex/AGENTS.md. That is one cause of the low RTK prefix rate measured on
GPT-6 lanes (M6c).

- build_args.py: codex_user_instructions() imports the checkout's
  tools/adoption/apply_codex_lane.py and returns agents_block(), the
  managed block of adoption/templates/codex.AGENTS.template.md (the top
  rule, rtk-ai/rtk v0.50.0 hooks/rtk-awareness-full.md verbatim, the RTK
  exactness exceptions). stage_lane_home writes it as codex-home/AGENTS.md
  and records agents_sha256 in staged.json. No copy of the text.
- tests: the staged AGENTS.md equals the template bytes and its sha256 is
  recorded; red before the change (no AGENTS.md).
- README (harness) lists the third file. recipes/README.md's OmniRoute
  "Workers" bullet no longer says the lane has no key filter (#393 added
  [shell_environment_policy.filters]) and names AGENTS.md.

Tests: 10 covering modules, 719 run, OK. validate.py passed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins added a commit that referenced this pull request Sep 27, 2026
@seathatflowsinourveins Scout @claude
Landscape sweep: the gateway lane home carries the host's Codex user …
a260a0f 
…instructions (AGENTS.md) (#418)

* Landscape sweep: the gateway lane home carries the host's Codex user instructions (AGENTS.md)

The OmniRoute lane (build_args.py --gpt6-provider omniroute) runs Codex with
CODEX_HOME set to a lane-local home holding only config.toml and the
stack-worker profile. Codex reads $CODEX_HOME/AGENTS.md as global
instructions, so the lane's GPT-6 got neither the repository's top rule nor
RTK's instructions, which the native lane's workers get from
~/.codex/AGENTS.md. That is one cause of the low RTK prefix rate measured on
GPT-6 lanes (M6c).

- build_args.py: codex_user_instructions() imports the checkout's
  tools/adoption/apply_codex_lane.py and returns agents_block(), the
  managed block of adoption/templates/codex.AGENTS.template.md (the top
  rule, rtk-ai/rtk v0.50.0 hooks/rtk-awareness-full.md verbatim, the RTK
  exactness exceptions). stage_lane_home writes it as codex-home/AGENTS.md
  and records agents_sha256 in staged.json. No copy of the text.
- tests: the staged AGENTS.md equals the template bytes and its sha256 is
  recorded; red before the change (no AGENTS.md).
- README (harness) lists the third file. recipes/README.md's OmniRoute
  "Workers" bullet no longer says the lane has no key filter (#393 added
  [shell_environment_policy.filters]) and names AGENTS.md.

Tests: 10 covering modules, 719 run, OK. validate.py passed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Landscape sweep #418 repair: restore the whole import path around the installer import

GPT-6 review of 2854514 (P3): importing tools/adoption/apply_codex_lane.py
runs its line 69, which prepends the repository root to sys.path, so
sys.path.pop(0) removed that root and left tools/adoption ahead of the
harness. codex_user_instructions() now snapshots and restores the full
sys.path. New test (installer unloaded first): red without the fix, green
with it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins seathatflowsinourveins mentioned this pull request Sep 27, 2026
Closed
seathatflowsinourveins pushed a commit that referenced this pull request Sep 27, 2026
Scout @claude
Codex worker lane: repair round for the required start-up review
bb35c5e 
Resolves the independent review's findings on 68c4a8d and records the
upstream tests that this host cannot run.

- Line citation: at openai/codex rust-v0.157.1, core/src/session/session.rs
  calls install_initial_mcp_runtime(...) on L1860-1866 and awaits it with
  `.await?` on L1867. The template, two test comments and the decision
  record now cite L1860-1867.
- Decision record, Problem: "Both exceed the grace" overstated the probe.
  The stdio probe times each server from its own launch, while the grace
  ends 1 s after Codex's first tool-list build (tool_catalog.rs L252-258).
  In the race test's failing-first runs the first request left 1.63 s and
  1.69 s after codex started. Only serena's slower starts (up to 2.24 s)
  are shown past that point. Codebase-memory's absence on the gateway
  route rests on the relayed peer measurement.
- Race test: the deferred-tool guidance proves only that some deferred
  tool was present. The fake endpoint now answers the first request with a
  code-mode exec call that lists ALL_TOOLS by name, as upstream's
  core/tests/suite/code_mode.rs run_code_mode_turn_with_builder (L264-293)
  and code_mode_exports_all_tools_metadata_for_namespaced_mcp_tools
  (L7133-7175) do. It reads the list from the second request's
  custom_tool_call_output. With the profile the first turn lists
  mcp__serena__fixture_serena and
  mcp__codebase_memory__fixture_codebase_memory. The relaxed control lists
  neither, and the two lists differ by exactly those names.
- Failing-first attribution: the native apply test's before run (real
  codex, rehearsal failed on codebase-memory) is the behavioural evidence.
  The synthetic ApplyFlowTests/RequiredStartTests before run failed with
  AttributeError on the three helpers that did not exist yet.
- Gateway profile template comment: since #393 the landscape sweep's lane
  home writes [shell_environment_policy.filters] OMNIROUTE_API_KEY =
  "exclude" (build_args.py L259-261 and L270-271). The template comment and
  the lane test's comment now say so, and the lane test checks that the
  staged filter equals the profile's.
- Decision record: lists the relevant upstream tests at rust-v0.157.1 and
  records them as not run, because this host has no Rust toolchain.

Evidence (codex-cli 0.157.1, TMPDIR=/var/tmp/claude-codex-startup):
- strengthened race test: exit 0, Ran 1 test, OK. On a copy of the tree
  whose template lacks the two required lines: exit 1, FAILED
  (failures=3), first request at 1.69 s, and the two first turns' tool
  lists were identical.
- tests.test_codex_worker_lane, tests.test_landscape_sweep_harness and
  tests.test_codex_lane together: exit 0, Ran 253 tests, OK (skipped=12).
manifests/evidence.json is not re-registered here; the coordinator does
that on replay.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@seathatflowsinourveins