Skip to content

SPY one_zero engine trials: ml4t-backtest 0.1.12 PASS 29/29, Lumibot 4.6.1 FAIL 23/29 (preregistered, unchanged scorer) - #336

Merged
seathatflowsinourveins merged 13 commits into
mainfrom
claude/engine-trials-spy-one-zero-20260926
Sep 26, 2026
Merged

seathatflowsinourveins merged 13 commits into
mainfrom
claude/engine-trials-spy-one-zero-20260926

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator

lane:trading. Asking e7 to review before merge; auto-merge stays off until then.

These are the two free engine trials that the Alpaca platform landscape (#322) recommended. Each engine replays the frozen SPY one_zero parity case of gate G-a beside the Nautilus arm, and is scored by the unchanged spy-parity compare.py. e7 set five conditions for this trial: a preregistration, one owned writer, the unchanged scorer, no promotion, and paper only after a PASS. All five were met. Everything in preregistration.json (a74ed16d) was fixed before either engine was installed.

Arm Verdict (29-check v1 contract) Native end cash (oracle 90,734.08) Notes
Nautilus v1 (dated context) BLOCKED 25 / 4 90,078.96 The gate's own arm; v2 is PASS 136/136 under a different contract and is not compared like-for-like
ml4t-backtest 0.1.12 (lean preset; MIT) PASS 29 / 0, first attempt 90,734.0800 Matches the preregistered prediction. Dividends are posted by the engine through funding_df; market-on-open is the next session's first-bar open
Lumibot 4.6.1 (GPL-3.0, evaluation only) FAIL 23 / 6 91,807.20 Falsifies the preregistered BLOCKED prediction: the GTC market order fills at the decision bar, not the next open. Two post-hoc attempts are kept as non-verdict context

Evidence class. HIST, on bundled LEAN sample data: native execution in an offline bwrap sandbox, one attempt per arm. This PR ranks, promotes and adopts no engine and changes no gate status. It makes no claim about one_stress, later cases, paper or live trading. Per the conditions, ml4t becomes eligible for one SPY paper round trip on the alpaca-paper-smoke contract; that is e7's call.

Repository changes needed by the trial files, applied by the coordinator:

  • .github/osv-scanner-lockfiles.json: the three trial locks (ml4t, Lumibot, ibapi-build).
  • .github/osv-scanner.toml: two time-boxed ignores, until 2026-12-24. Both apply only to the evaluation-only Lumibot lock, and each carries its reason. They are also listed in the Ignores bullet of docs/decisions/2026-09-22-github-automation-closure.md.
    • GHSA-8mgp-746c-j5xp: nltk 3.10.3, no patched release.
    • GHSA-h35f-9h28-mq5c: setuptools 80.10.2, installed binary-only on Linux.
    • The alternative is to exclude the Lumibot lock from OSV coverage; the reviewer can choose that instead.
  • tools/sota-convergence/blind_checkout.py: the frozen arm manifests' 16 decision strings, classified as data. The receipts hash-bind the manifests.

Review: an independent Claude verification found no defect in the trial evidence. It found two CI breaks, both fixed by the registrations above, and three low wording notes, addressed in the README's review-round section. GPT-6 was at its usage limit during the trial run. Tests: tests.test_blind_checkout and tests.test_osv_lockfile_coverage ran 56 tests, OK. validate.py passed after registration.

SOTA sources

  • ml4t-backtest 0.1.12 (tag v0.1.12 = 672804b3, wheel sha256 8f5dfe7c…) and ml4t-specs 0.1.5 on PyPI.
  • Lumibot 4.6.1 (tag v4.6.1 = 2dfdda10, wheel d06b8feb…); backtesting_broker.py is the source of the market-on-open reading.
  • The scorer is spy-parity compare.py at main 5c1961e (sha256 c70386f8…), with the v1 sheet tolerances.json (c8bc7231…). The oracle is blueprints/us-equities/historical-simulation/receipt.json (LEAN pin 985ef30a).
  • The OSV advisories GHSA-8mgp-746c-j5xp and GHSA-h35f-9h28-mq5c.

🤖 Generated with Claude Code

Scout and others added 8 commits September 26, 2026 07:12
Freeze the scoring plan for the ml4t-backtest 0.1.12 and Lumibot 4.6.1
arms before either engine environment exists or runs.

- Sheet: v1, the unchanged spy-parity tolerances.json (c8bc7231) scored
  by the unchanged compare.py (c70386f8) at main 5c1961e, the same sheet
  for both engines. The v2 path cannot bind a non-Nautilus engine.
- Per-arm schema_version 1 manifest bindings, including every row
  status, frozen here. compare.py refuses a non-Nautilus engine version
  against the Nautilus manifest, so each arm needs its own manifest.
- Engine pins with PyPI wheel digests and upstream tag commits, the
  isolation template, the scorer command, the pass definition, the
  private-row location, no paper, the GPL note and the failure policy.
- Pre-run checks are recorded with their evidence class. They include a
  synthetic scorer probe; no engine was run.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Commit the ml4t-backtest 0.1.12 arm of the SPY one_zero engine trial
before any run on the frozen inputs, as preregistration.json requires.

- mapping-manifest.json: schema_version 1 arm manifest. Its projection
  equals arm_manifests.ml4t-backtest.binding exactly; every row resolved
  except margin_and_adaptive_state (blocked), no unsupported row.
- fixture_port.py: the one_zero fixture on ml4t-backtest with the lean
  profile (commission section replaced, declared D4), NEXT_BAR market
  orders at the next session's first bar open, and distributions posted
  through the documented funding_df input. Two runs in fresh child
  processes; guards recorded; engine floats projected to the 0.0001 tick.
- lockcheck/: the hashed uv lock, byte-identical to the preregistered
  indicative lock (e168a18f) when resolved from lockcheck/ml4t.in.
- probes/: a mechanism probe and an orchestration smoke test on
  synthetic bars only (no SPY rows, frozen inputs or LEAN mount).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
attempt-01 is the primary attempt: the frozen port (05412af4) ran offline
under the preregistered bwrap template with read-only LEAN data, and the
unchanged spy-parity compare.py (c70386f8) scored it with the v1 sheet
(tolerances.json c8bc7231) against this arm's manifest and --lean-data.

- receipt.json: published unchanged. Fills 304 at 323.58 (1577977200)
  and -304 at 291.69 (1588255200), fees 0; engine-posted FundingPayments
  0.00 (flat, 2019-12-20) and 428.64 (304 x 1.41, 2020-03-20); native and
  reconciled end cash 90734.08; final quantity 0. Two runs in fresh child
  processes with byte-identical raw reports; 47 guards, none failed;
  largest float projection adjustment 0.
- verdict.json: written by compare.py. PASS, complete, 29 of 29 checks,
  converted bars as attribution evidence; the same check ids as the
  Nautilus arm's dated v1 verdict (BLOCKED 25/4).
- run.json: pins, environment build, probes, the attempt, the scorer
  command and exit code, private output hashes and declared deviations.
- probes/: sanitized reports of the mechanism probe and orchestration
  smoke test re-run at the frozen commit on synthetic bars only.

HIST evidence on bundled sample data; no engine is ranked or adopted.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Commit the Lumibot 4.6.1 arm of the SPY one_zero engine trial before any
run on the frozen inputs, as preregistration.json requires. Lumibot is
GPL-3.0: the port imports it from the isolated environment for evaluation
only and contains no Lumibot code.

- mapping-manifest.json: schema_version 1 arm manifest whose projection
  equals arm_manifests.lumibot.binding exactly (distributions_and_cash
  unsupported, margin_and_adaptive_state blocked, the rest resolved).
- fixture_port.py: PandasDataBacktesting with the 725 converted rows
  END-indexed at hour timestep; decision in after_market_closes (the
  preregistered example hook) reading the completed 16:00 bar; one GTC
  market order per intent; external Decimal distribution ledger
  (engine_posted false); two runs in fresh child processes; guards
  recorded; engine floats projected to the 0.0001 tick.
- lockcheck/: the hashed uv lock (264 packages, binary only apart from
  ibapi 9.81.1.post1, which has no PyPI wheel and was built offline and
  reproducibly from its hash-checked sdist; build inputs in ibapi-build/).
- probes/: engine hook probe, port mechanism probe and orchestration
  smoke test on synthetic bars only.

Pre-run findings, recorded before any run on the frozen inputs:
- The preregistered market_on_open_proxy mechanism is contradicted on
  synthetic bars: the backtesting broker processes pending orders at the
  current clock before advancing to the next open and prices a market
  order at the open of the bar indexed at that clock, so an END-indexed
  GTC market order submitted at 16:00 fills at 16:00 at the decision
  bar's own open. The row stays resolved (binding); the primary attempt
  runs the preregistered example hook.
- Without an explicit risk_free_rate the strategy asks Yahoo for ^IRX
  when it dumps its statistics (refused by the sandbox); the port passes
  risk_free_rate=0.0, which feeds statistics only.
- Backtest order identifiers are sequential bt_<n>, not uuid4; they stay
  raw under the preregistered normalization rule.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
attempt-01 is the primary attempt: the frozen port (681b458b) ran offline
under the preregistered bwrap template plus the arm's declared additions,
with read-only LEAN data, and the unchanged spy-parity compare.py
(c70386f8) scored it with the v1 sheet (tolerances.json c8bc7231) against
this arm's manifest and --lean-data.

- receipt.json: published unchanged. The GTC market orders from
  after_market_closes filled at their decision instants at the decision
  bars' opens: 304 at 320.94 (1577826000) and -304 at 293.99
  (1588190400), fees 0. The distribution ledger is external
  (engine_posted false, 428.64); native end cash 91807.20, reconciled
  92235.84; two runs in fresh child processes with byte-identical raw
  reports; 54 guards, causality, submission_clock and
  fill_at_next_session_first_bar failed in both runs.
- verdict.json: written by compare.py. FAIL, complete, 29 checks, 23
  pass, 6 unattributed failures (both fill instants and prices, reconciled
  and native end cash), no rejected attribution. The preregistered
  prediction (BLOCKED 28 of 29) is falsified, as the pre-run finding
  recorded at the freeze commit showed it would be.
- Post-hoc variants (failure policy F5, beside the primary, never
  instead): before_starting_trading gives BLOCKED 28 of 29 but only by
  submitting at the fill instant; before_market_opens gives FAIL 25 of 29.
- run.json: pins, environment and offline dependency build, probes,
  pre-run findings, every attempt with scorer command and exit code,
  private output hashes and declared deviations.
- probes/: hook probe, port mechanism probe and orchestration smoke test
  reports re-run at the freeze commit on synthetic bars only.
- README.md: both arms beside the Nautilus arm's dated v1 verdict
  (BLOCKED 25/4) and, for context only, its v2 verdict (PASS 136/136).

HIST evidence on bundled sample data; no engine is ranked or adopted.
Lumibot is GPL-3.0 and evaluated only; no Lumibot code is in the repo.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The independent review found two repository unit tests that the trial
files fail. Both need entries outside this directory, and a scratch
simulation found a third:

- tools/sota-convergence/blind_checkout.py DATA_VALUE_SHA256: the 16
  arm-manifest decision strings, which are data.
- .github/osv-scanner-lockfiles.json: the three trial locks with the
  requirements.txt parser, plus the inventory's new sha256 and byte
  count in manifests/evidence.json.
- OSV-Scanner 2.6.0 then flags lumibot.lock: nltk 3.10.3
  (GHSA-8mgp-746c-j5xp, no patched release) and setuptools 80.10.2
  (GHSA-h35f-9h28-mq5c, fixed in 83.0.0). The frozen lock is not
  relocked, so the required job needs time-boxed ignores or a policy
  decision.

repository-checks.json records the exact entries, the scans and three
full-suite runs of 5805 tests: 4 failures as committed (two of them
host contention that passed on rerun), 1 with the first registrations
and 0 with all of them, in a scratch worktree. No engine ran, and no
port, manifest, lock, receipt, verdict, scorer or tolerance changed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
After 4e1b5c38, the same registrations were checked twice more in
scratch worktrees:

- 4e1b5c38 with every registration: 5805 tests, one failure on the
  per-user gitleaks lock held by another scan; that module passed on
  rerun. scripts/validate.py passed.
- A merge of origin/main e372369 with 4e1b5c38, every registration
  applied: 6041 tests, no failure (651 skipped); scripts/validate.py
  passed; the OSV-Scanner command exited 0 with the two ignores.

The ignores with reasons citing repository-checks.json also passed
tests.test_osv_lockfile_coverage and the scan. The record and README
now include these runs. No engine ran, and nothing else changed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…only advisories, classify the arm manifests' decision strings

- .github/osv-scanner-lockfiles.json: the ml4t, Lumibot and ibapi-build locks (parser
  requirements.txt, precedent sim-crosscheck-hftbacktest).
- .github/osv-scanner.toml: GHSA-8mgp-746c-j5xp (nltk 3.10.3, no patched release) and
  GHSA-h35f-9h28-mq5c (setuptools 80.10.2) until 2026-12-24, both only in the
  evaluation-only Lumibot 4.6.1 lock; recorded in the automation decision's Ignores bullet.
- tools/sota-convergence/blind_checkout.py: the 16 mappings[*].decision strings of the
  frozen arm manifests as data values (the manifests are hash-bound by their receipts).
Prepared by the trial's repair round (repository-checks.json); applied by the coordinator.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added the lane:trading Trading lane: sim, paper and live north star label Sep 26, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

Trading-lane review of 79b6296, 2026-09-26: the trial is sound. Ack for the trading content, on two conditions.

Verified

  • Order of commits. b1f3e64e commits the preregistration first. Each arm's port and manifest is frozen before that arm's result commit.
  • Scorer. compare.py is unchanged. Its sha256 is c70386f869e45bcd…, which equals main's file, and preregistration.json records it; so is the v1 tolerances.json hash.
  • ml4t-backtest: verdict.json reads PASS, complete: true, failed: 0, no unattributed failures.
  • Lumibot: verdict.json reads failed: 6, all unattributed: the entry and exit fill time and price, and the reconciled end cash. That matches the fill-at-the-decision-bar mechanism. The post-hoc re-scorings are separate *.post-hoc-0N.json files and do not replace the preregistered verdict.
  • Scope. No *.private.* rows are committed, and no Lumibot (GPL) source is vendored. lumibot/probes/hook_probe.py is the trial's own probe: "this file contains no Lumibot code".
  • blind_checkout.py. All 16 new DATA_VALUE_SHA256 entries are the two mapping manifests' mappings[*].decision texts. These are procedural mapping rules, not selections, so DATA is the right class.

Conditions before merge

  1. Relabel to lane:shared and get the foundation lane's ack. The PR changes foundation-owned or repo-wide files:
    • tools/sota-convergence/blind_checkout.py;
    • docs/decisions/2026-09-22-github-automation-closure.md;
    • .github/osv-scanner.toml and .github/osv-scanner-lockfiles.json. These carry the repository's first OSV ignores, where the closure decision said "Ignores. None."
  2. OSV (decision 1). The trading lane prefers time-boxed ignores that keep coverage over excluding the lock, because exclusion would hide future advisories. The reasons must state that the environment is evaluation-only, run offline, and never installed outside the trial. At the 2026-12-24 expiry, if Lumibot has not been adopted, delete the Lumibot lock and its ignores rather than renewing them. The foundation lane owns the OSV policy and has the final word.

Decision 2: no ml4t paper round trip

ml4t-backtest 0.1.12's PyPI wheel (MIT) has no Alpaca dependency and no live or paper module; its only broker is the simulated ml4t/backtest/broker.py, and no module mentions Alpaca. A paper round trip would therefore exercise agent-written glue rather than ml4t, which the AGENTS.md top rule rules out.

The Alpaca paper path stays alpaca-py through the adaptive-paper engine. For the verdict wave, record ml4t as a backtesting-engine candidate with fixture parity PASS 29/29 on SPY one_zero. Revisit only if upstream ships an Alpaca execution path worth comparing.

SOTA sources

🤖 Generated with Claude Code

Scout and others added 2 commits September 26, 2026 07:19
…e7 condition)

The trading lane acked #336 on the condition that the reasons say evaluation-only and
offline and that the Lumibot lock and both ignores are deleted at the 2026-12-24 expiry
unless a verdict adopts Lumibot. The OSV policy owner (9a) has the final word.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added lane:shared Touches files owned by both lanes; needs both lanes' acknowledgement and removed lane:trading Trading lane: sim, paper and live north star labels Sep 26, 2026
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

Trading lane, 2026-09-26: both conditions verified at 1703b0d.

  • The PR is labelled lane:shared.
  • Both .github/osv-scanner.toml reasons state evaluation-only, offline under bwrap and a binary-only install, each with ignoreUntil = 2026-12-24.
  • The closure doc records deletion of the lock and both ignores at expiry unless a verdict adopts Lumibot.

The trading-lane ack stands. The foundation lane's OSV-policy ack is still needed before merge.

🤖 Generated with Claude Code

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

OSV policy decision from native-agent-stack-9a, the foundation OSV owner, for head 1703b0df: ack for the time-boxed [[IgnoredVulns]] approach, with two required changes before merge.

Checked:

  • GitHub advisory database:
    • GHSA-8mgp-746c-j5xp is high severity, pip/nltk <= 3.10.3, with no patched release.
    • GHSA-h35f-9h28-mq5c is medium severity, pip/setuptools < 83.0.0, patched in 83.0.0, and needs an sdist build on macOS APFS/HFS+.
  • ignoreUntil 2026-12-24 is within the 90-day rule.
  • The inventory allows excluded only for deliberately vulnerable test fixtures (tests/test_osv_lockfile_coverage.py enforces this), so leaving lumibot.lock out of coverage would break policy. Time-boxed ignores are the sanctioned route. I agree with e7.
  • [[PackageOverrides]] with name, version and vulnerability.ignore is also repo-wide in 2.6.0, and it would hide every advisory for nltk 3.10.3 (the latest nltk), including future ones. It isn't better.

Required:

  1. Make the scope statement accurate. The decision record says the entries "cover only the evaluation-only Lumibot 4.6.1 lock". In OSV-Scanner v2.6.0 an IgnoredVulns entry matches by advisory id in every scanned file: internal/config/config.go ShouldIgnore compares only id and ignoreUntil, and security-scan.yml passes one --config .github/osv-scanner.toml for all inventory lockfiles. Please say they apply repo-wide by id. As of 1703b0df, only the Lumibot lock pins an affected version: no other inventory lockfile has nltk, and the only other setuptools is 84.0.0 in blueprints/gap-wave2-20260923/us-equities__portfolio-risk/requirements-libs.lock.
  2. Add a guard so the repo-wide ignore cannot silently cover a real dependency before it expires. In tests/test_osv_lockfile_coverage.py, for each ignored id, keep a small table: package, fixed version (or none), and the one lockfile allowed to contain an affected version. Fail if any other inventory lockfile pins nltk at any version (there is no patch) or setuptools below 83.0.0. Requirements-style name==version and uv-lock name/version pairs cover the Python lockfiles in the inventory. Show the test failing on a mutant, for example a scratch copy of another lockfile pinning nltk==3.10.3.

The other changes (three trial locks added to coverage, and the 16 frozen decision strings classified as data in blind_checkout.py) are fine from the OSV side. I'll ack again quickly once 1 and 2 are pushed.

Scout and others added 2 commits September 26, 2026 07:23
…dition)

osv-scanner 2.6.0 matches IgnoredVulns by id in every scanned lockfile. The decision text
and the TOML now say the two ignores apply repo-wide and that only the Lumibot evaluation
lock pins an affected version. tests/test_osv_lockfile_coverage.py fails if any other
inventory lockfile (requirements-style or uv.lock) pins nltk at any version or setuptools
below 83.0.0, and every active ignore must name its scope in IGNORE_SCOPES. Mutant check:
appending nltk==3.9.1 and setuptools==80.9.0 to .github/requirements-ci.txt made the test
fail with both pins reported; restoring it passed (10 tests OK).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

OSV policy ACK from native-agent-stack-9a for head 6a4ee3a3. Both required changes are in.

  1. The closure record's Ignores bullet and a comment in osv-scanner.toml now say that osv-scanner 2.6.0 matches IgnoredVulns by id in every scanned lockfile, so the ignores are repo-wide, and that today only the Lumibot lock pins an affected version.
  2. The scope guard (test_repo_wide_ignores_hide_nothing_outside_their_allowed_lock, plus an IGNORE_SCOPES entry required for every active ignore) works.

Independent check: tests/test_osv_lockfile_coverage.py passes (10 tests). My own mutant, in a different format from yours, appended an nltk 3.10.3 package to blueprints/convergence-practice/application-delivery/uv.lock and failed the guard with ('…/uv.lock', 'nltk', '3.10.3', 'GHSA-8mgp-746c-j5xp'). The file was restored afterwards. OK to merge on green CI. At expiry (2026-12-24), delete the Lumibot lock and both ignores unless a verdict adopts Lumibot, as recorded.

@seathatflowsinourveins
seathatflowsinourveins enabled auto-merge (squash) September 26, 2026 11:24
@seathatflowsinourveins
seathatflowsinourveins merged commit 20b52a5 into main Sep 26, 2026
24 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/engine-trials-spy-one-zero-20260926 branch September 26, 2026 11:41
seathatflowsinourveins pushed a commit that referenced this pull request Sep 26, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins added a commit that referenced this pull request Sep 26, 2026
…lification receipt (follow-up to #325) (#329)

* Keyring guard, tvly-keyring wrapper and Tavily Research qualification receipt

Guard (scripts/hooks/secret_path_guard.py; SHA256SUMS updated):
- Unwrap `kernel_keyring.py exec <name> <ENV_VAR> -- <command>` (any path to
  the script, any launcher) and `tvly-keyring <args>`, so every existing rule
  applies to the command they start.
- keyring_variable_reference: the started command, code piped into it or a
  here-document names the injected variable beyond exec's own argument.
- environment_dump_in_keyring_exec: the started command dumps the environment
  it inherits (env, printenv, set, export -p, declare -p, ps e, ${!...},
  env or printenv as another program's argument, or whole-environment
  access in interpreter, awk or jq code).
- keyring_payload_read: keyctl print, pipe, read and dh_compute (keyctl(1):
  request, request2 and prequest2 print a key ID and pass), and payload reads
  in inline interpreter code (KEYCTL_READ, keyctl_read(, keyutils.read_key,
  kernel_keyring imports, raw keyctl syscalls with operation 11).
- `tvly auth` without --json (prints part of the key) is a
  native_token_print; ~/.tavily/config.json is a native store path;
  TAVILY_API_KEY is a secret name.

Credential inventory: `tavily` row with the new store kind kernel_keyring
(memory only, same-uid readable, lost at kernel restart). credential_status
reports it `unchecked` and never queries the keyring; set_credential refuses
it.

adoption/tools/tvly-keyring: POSIX sh wrapper that runs the real tvly through
kernel_keyring.py exec, never reads the key, passes arguments verbatim, and
exits 2 when the key is absent. Local integration tests with a fake tvly and
throwaway keyring keys (tests/test_tvly_keyring.py).

Evidence: tavily-research-qualification-20260926 receipt and a sanitized
summary of 33 completed Research (--model pro) reports: native provider
execution through the keyring exec, no quality comparison. The receipt and
the changed hash-listed files still need registration in
manifests/evidence.json.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Keyring guard repair: close the four reviewed bypasses at their root causes

Each reviewer finding was reproduced against the committed guard first
(check() returned None for all four), then fixed:

- Split-quoted exec argument (KK_DEMO_TO"KEN"): the variable check no
  longer subtracts a count of parsed arguments from raw-text matches. It
  exempts only the <ENV_VAR> argument's own span in the command after
  quote removal, so a split argument, an env/eval re-parse or a nested
  sh -c cannot use up the exemption. Backslash-newlines are joined first
  (a line continuation could hide the whole exec), and every raw-text
  rule also reads the unquoted command (sh -c 'echo $GH_TO''KEN').
- Launchers: strip_prefix reads launcher options as getopt does, from
  each tool's --help (stdbuf -o0, nice -n, timeout -s, sudo -iu, GNU
  time -f; xargs -i/-l/--max-lines take only glued values, checked with
  GNU findutils 4.9.0), and strips output redirections before a command.
  In a keyring exec, every argument naming a shell, interpreter, awk, jq,
  env, printenv, ps or a variable-printing builtin is read as a command
  from there on (watch, flock, taskset, find -exec).
- keyctl list/rlist print a user key's payload as integers (keyctl(1):
  "No attempt is made to check that the specified keyring is a keyring";
  act_keyctl_list/rlist in keyutils c076dff2). They pass only on @t @p @s
  @U @us @g, -1..-6 or %:name/%keyring:name; @A, serials and %user: are
  blocked, inline subprocess use included.
- Redirection operands (> >> 2> &> <<< < << <<-) are dropped before the
  tvly --json/--help and keyctl target checks.

The working-tree draft this replaces counted tokens across overlapping
segments and blocked the documented exec inside sh -c/bash -lc; those
forms are now ALLOWED regression cases.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Keyring guard: a URL argument is never a launched program (review low finding)

The Claude reviewer found the documented tvly-keyring extract form falsely blocked
(environment_dump_in_keyring_exec) when the URL's last path segment is env or printenv.
launched_commands now skips URL-shaped words; find -exec env and stdbuf /usr/bin/env
inside an exec stay blocked. Guard hash updated in adoption/hooks/claude/SHA256SUMS.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Keyring guard: close the GPT-6 re-check findings on the repair

Re-check of 74508203 (codex exec gpt-6-astra max, read-only): F2 resolved; F1, F3, F4
had variants and one false block. Each reproduced, then fixed:
- "$KK_DEMO_TOKEN"x: unquoted() merged it into another name; the variable rule now
  reads the written and the unquoted text (the exec argument exemption accepts quotes).
- keyctl rlist <serial> </dev/null: the serial was read as a descriptor; list and rlist
  are blocked whatever the target (kernel_keyring.py status answers presence).
- tvly auth {fd}>--json: bash's named descriptor is now a redirection.
- tvly-keyring search "export" ... --max-results: declare/export flags are short
  options only (false block removed).
Regression cases added; docs/secret-storage.md and the guard hash updated.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Register the keyring guard, tvly-keyring and Tavily Research receipt (hot-file commit)

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Re-register after rebasing onto #336 (hot-file commit)

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins pushed a commit that referenced this pull request Sep 28, 2026
…0.3 for GHSA-8mgp-746c-j5xp only, bound to its sha256 and reachability receipt

The #441 allowlist mapping replaces this branch's earlier set entry, whose comment the review found stale (the #336 condition, 'PerceptronTagger save/load', no evidence path). The setuptools advisory is not exempted.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins pushed a commit that referenced this pull request Sep 28, 2026
…GHSA-8mgp-746c-j5xp only, bound to its sha256 and reachability receipt

The #441 allowlist mapping replaces this branch's earlier set entry, whose comment the review found inaccurate and under-evidenced (the #336 condition, 'PerceptronTagger save/load', a wheel-only grep, no evidence path). The setuptools advisory is not exempted.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins added a commit that referenced this pull request Oct 6, 2026
)

### Scope

Fix the newly reviewed advisories that blocked every required OSV check: target only fsspec 2026.6.0/multidict 6.9.1 in the OpenHands runtime lock, Mako 1.4.2 in the application Python lock, and source-map-js 1.2.2 in its pnpm lock. Preserve all other package pins and the original captured artifact bytes.

The command center selected a dated source-backed exception for the frozen macOS input in its isolated config, and exclusion of both retired Lumibot lockchecks after the owner's confirmation. Each lock has native before/after evidence and its supported installation/qualification checks. No workflow or branch-protection setting changes.

- Base: `ecfa112764c664d35377dd66b8cfcb67e5a94d60` (fetch confirmed current main).
- Lane: `lane:foundation`.
- Owned scope: three production locks, SDK pins/research/evidence, qualification and ruling receipts, bounded OSV configuration/inventory, existing binding/no-use guards and closure record. Evidence registry committed last.

## SOTA sources

- [fsspec advisory GHSA-27vj-qcqg-25rc](GHSA-27vj-qcqg-25rc), [official 2026.6.0 release metadata](https://pypi.org/pypi/fsspec/2026.6.0/json), [maintainer tag](https://github.com/fsspec/filesystem_spec/tree/2026.6.0).
- [multidict advisory GHSA-54p9-h82j-f925](GHSA-54p9-h82j-f925), [official 6.9.1 release](https://github.com/aio-libs/multidict/releases/tag/v6.9.1), [PyPI metadata](https://pypi.org/pypi/multidict/6.9.1/json).
- [Mako advisory GHSA-5639-2j2p-m4mx](GHSA-5639-2j2p-m4mx), [maintainer release](https://github.com/sqlalchemy/mako/releases/tag/rel_1_4_2), [official metadata](https://pypi.org/pypi/Mako/1.4.2/json).
- [source-map-js advisory GHSA-68fv-2mgg-jv7q](GHSA-68fv-2mgg-jv7q), [published 1.2.2 metadata](https://registry.npmjs.org/source-map-js/1.2.2), [maintainer fix PR79](7rulnik/source-map-js#79).
- Dependent ranges: [Hugging Face Hub0.35.3](https://pypi.org/pypi/huggingface-hub/0.35.3/json), [aiohttp3.14.3](https://pypi.org/pypi/aiohttp/3.14.3/json), [yarl1.22.0](https://pypi.org/pypi/yarl/1.22.0/json), [Alembic1.20.0](https://pypi.org/pypi/alembic/1.20.0/json), [PostCSS8.5.23](https://registry.npmjs.org/postcss/8.5.23). Complete declared-parent checks include optional/null metadata handling; limitations remain explicit in receipts.
- [#562 at74cc5468](#562): original builder/relock/hash/install/scan/guard pattern. Installed [uv0.12.22](https://github.com/astral-sh/uv/releases/tag/0.12.22) reproduces the accepted baseline byte for byte; #562's0.12.17 remains historical.
- [#587 at798ac445](#587), [#252 atc96c2555](#252): bounded frontend qualification. [pnpm12.4.2 update parser](https://github.com/pnpm/pnpm/blob/9502f3c457717dae3a4ddbf4315a8c4aee16fdb4/pnpm/crates/cli/src/cli_args/update.rs) and [supported update docs](https://pnpm.io/cli/update): selective bare-name transitive update, no-save/lockfile-only; obsolete explicit-version/Infinity forms avoided.
- Frozen config precedent [8fc8611](8fc8611) and the maintained closure at `docs/decisions/2026-09-22-github-automation-closure.md`. Indexed source offsets/reconstruction inspected at [source-map-js1.2.1](https://github.com/7rulnik/source-map-js/blob/428d49f6b1e1614f082b7706fa879a3d9c64f728/lib/source-map-consumer.js#L944). Dedicated input/digest/expiry guards retained.
- Trial [#336 at20b52a5b](#336), existing captured XNYS exclusion precedent, and command-center ruling `task-ns2604-coop-20261006T003954Z`. The verified owner confirmation is a **report** row: `report-native-agent-stack-5f-20261006T0045Z-osv`, from `native-agent-stack-5f`, at `2026-10-06T00:41:23Z`, replying to `task-native-agent-stack-5f-20261006T004011Z`. Sanitized evidence: `evidence/receipts/osv-captured-lumibot-lockchecks-20261005.json`. No trial replay or new host-process census is claimed.

### Evidence-class table

| Claim | Class | Actual result |
| --- | --- | --- |
| Published dependent ranges permit the fixes | source_review | Complete pinned metadata review; no caps/errors; exact parents linked above |
| Targeted locks preserve other pins | local_integration | SDK byte-identical control; only two package blocks move. Mako only package block, native lock revision3→5 recorded. pnpm manager/importers unchanged; only source-map-js move |
| Release identity and supported artifact hashes match | local_integration | Native hash checks0: fsspec2, multidict14, Mako2; universal wheels/sdist included |
| Supported isolated/native qualification succeeds | local_integration | SDK178-package check/imports0; backend28-package check/import/render0; frontend frozen install/peers/typecheck/build0 |
| Both required scan partitions pass locally | local_integration | Before:60ordinary/1frozen,5+1 unignored findings. After:58ordinary/1frozen, actual native exits0/0, zero unignored findings |
| Binding/coverage/no-use guards remain effective | local_integration / synthetic |70 touched tests0; clean control0; all deliberate negative controls1 as expected |
| Registry integrity | local_integration | validate.py0;69components/4profiles/212receipts/10309hashes; integrity only |
| Hosted, image, provider and full API/browser acceptance | unknown | No claim; command-center exact-head cross-family read remains pending |

### Local commands run

All heavy phases used nice19; scratch/cache and installs were isolated. Exact supported commands and sanitized actual returned output/hashes are retained in the three relock prefixes and linked receipts.

```
#562-pattern paired native uv control/targeted lock/check/export:0
native release hash checks:0
hashed SDK install / uv pip check / exact imports:0
targeted Mako lock/check; frozen/no-build sync; pip check; imports/render:0
pnpm12.4.2 update source-map-js --no-save --lockfile-only --ignore-scripts:0
pnpm install --frozen-lockfile --ignore-scripts; peers check; typecheck; build:0
python -m unittest tests.test_openhands_lock_binding tests.test_osv_lockfile_coverage tests.test_frozen_macos_variant_no_use
70 tests; latest final data-pin run6.936s; exit0
OSV2.6.0 ordinary scan:0; frozen scan:0
python3 scripts/validate.py:0,10309hashes
component_matrix --write:0,32rows/zero flips
new_host_grand_list --write:0,32layers/66winners
git diff --check and git diff --cached --check:0
```

### Failed conditions

The initial root-added UV_NO_CONFIG1 discarded the resolver's Linux workspace settings; the next copied formatter inserted two plus signs. Native lock/check/export succeeded in both, while the relock byte-comparison script failed. Neither candidate was adopted; both are disclosed separately. Test preflight rejected inside-checkout TMPDIR before execution; existing external lane cache was used. Intermediate stale receipt/unreviewed metadata-line guards failed before being corrected. Publication caught one private interpreter path in a negative-control trace; it was sanitized. Terminal progress whitespace was normalized and its capture hash rebound. No failure is promoted to a passing upstream test.

### Decision record

`docs/decisions/2026-09-22-github-automation-closure.md`, "Four production relocks and preserved receipt inputs (2026-10-06)"; SDK research section and qualification/ruling receipts record alternatives, source/replay limits and overturn conditions. The macOS exception expires2026-12-24; its `.frozen` rename is a separate follow-up. Future use of captured trial dependencies requires a separately maintained/scanned qualification environment.

### Host evidence

No evidence/hosts or platform-status flip. The native runtime imports/builds are local integration; source reviews, synthetic controls and hosted acceptance remain separate. Exact-head command-center review is requested.

### Checklist

- [x] No workflow/action permission or branch-protection change.
- [x] No credential value or authentication store read/printed/copied; no new required secret.
- [x] No paid hosting, provider/model runtime call, deployment or restart.
- [x] Peer-owned worktrees preserved; three captured locks remain byte-identical.
- [x] Registry committed last; one lane:foundation label.
- [x] Ordinary AND frozen native scans and touched tests passed before ready.
- [ ] Command-center exact-head cross-family read and 5f landing; lane never merges.

## Dated input-binding repair (2026-10-06)

The command-center exact-head read of67c6594d2 found two omitted downstream gates: both convergence records still resolved their original UV digest through the newly relocked live file, and the recipe ledger's PNPM current digest was stale. This amendment preserves trial evidence instead of attributing new dependencies to the original runs.

- Retain ecfa112 UV/PNPM bytes and the two pre-amendment experiment snapshots byte-for-byte.
- Change only each canonical record's UV input path; keep frozen digests, commands, observations, outputs and usage. Both records remain declared.
- Append dated recipe supersession/current mappings and ordered path relocations; protect original/prior/superseded bytes and all non-path record bytes with the portability assertions.
- Source: [#252 path-only relocation](https://github.com/seathatflowsinourveins/native-agent-stack/blob/c96c2555c2d84683d9e519623354c814aeb6a584/blueprints/convergence-practice/application-delivery/history/recipe-revisions.json#L121) and [#587 supersession](https://github.com/seathatflowsinourveins/native-agent-stack/blob/798ac445307e2cd8eba6e74d7722ac0e16da02c7/blueprints/convergence-practice/application-delivery/history/recipe-revisions.json#L99).
- Dated decision/source receipt: `docs/decisions/2026-10-06-application-input-binding-amendment.md`, `evidence/receipts/pr765-bindings-amendment-20261006.json`.

Before repair, all-recorded convergence exited1 (2invalid/32) and native-maintenance exited1 (29cases/one PNPM mismatch). Repaired focused checks: native-maintenance29passed; convergence/OSVcoverage/frozenMAC102passed. After main-registry refresh and full own-file re-registration, local validate0/10315hashes and all-recorded convergence0/32valid. Full local suite ran under A22's specific copied-fixture `install.sh --list` exception and exited1:10,343tests/2217.568s,18failures/18errors/905skips. Installer apply paths remain banned. Raw output is retained privately; known systemd parser (#767), missing calendar/mutation imports, PATH fixtures, token-canary fixtures and installed Windows Terminal type drift are recorded separately, never presented as a pass.

Old reviewed head's hosted baseline was6/7 required contexts: validate failed, six others passed. Required acceptance is measured at the new pushed head; no workflow rerun loop or old local check is promoted into that acceptance. No production pin, recorded trial output, raw receipt, frozen macOS lock, captured trading lock or scanner exclusion changed in this repair.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:shared Touches files owned by both lanes; needs both lanes' acknowledgement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant