PR-4: trading north-star gates, card reconciliation and the sim → paper → live ladder - #63

engines-strategies.json/.md: nautilustrader moves to default as the
selected destination runtime (native_proven, pinned commit
1b0a49d2792a9432a3aca3fcb617ce7a630d905e), with the SPY/LEAN parity gate
G-a BLOCKED note and the dividend SimulationModule ownership recorded.
lean stays default but reframed as the frozen historical comparator, not
the runtime. Adds nautilus-ibkr-adapter (source_review, local_broker_
acceptance not_established) and adaptive-paper-alpaca-adapter
(source_review, synthetic capacity fixture cited, no live broker fills
claimed).

agents-operations.{json,md}: grype notes the pending 2026-09-22 rc5/
Alpaca-adapter scan receipt; openbao notes IBKR as selected live-primary,
unaccepted.

Regenerates decision-index.json and bumps manifest.json repository_entries
(152->154) and unique_catalog_repositories (147->148) plus the grand_index_*
counts via scripts/catalog_decisions.py --write.

tools/sota-convergence/reconciliations-20260922.json: adds the 9
reconciliation entries for this wave (backtesting-engine card_reconciled,
2x execution-broker card_added, and 6 pending-sibling entries for market
data, security scan/decision, observability and data-quality gates).

tests/test_catalogs.py: extends with Pr4CardsReconciliationTests covering
the new/changed cards, the reconciliation entries and the manifest counts.

manifests/evidence.json: rehashed the 8 changed catalog/test files.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…daptive-paper metrics exporter

metrics.py serves Prometheus text-format /metrics on 127.0.0.1:18890, reading
only the durable adaptive-paper ledger.sqlite3 (opened file:...?mode=ro) and
its sibling trial.json; it never imports safety.Ledger, opens credentials, or
talks to the broker. It exports paper_trial_active, paper_needs_attention,
paper_reconciliation_status{result}, paper_order_state_divergence_total,
paper_ledger_frozen{reason}, and paper_request_budget_{remaining,limit}{kind},
and documents (rather than invents) the two fields the current ledger schema
cannot durably support: paper_reconciliation_last_success_timestamp_seconds
and paper_request_budget_wait_exceeded_total.

The observability backend templates add scrape job adaptive-paper, the
equities-broker-path Prometheus rule group (EquitiesOrderStateDivergence,
EquitiesReconciliationFailed, EquitiesRequestBudgetExhausted,
EquitiesLedgerFrozen, EquitiesPaperMetricsMissing), and an explicit
Alertmanager route for scope=equities-broker to the existing local-ntfy
receiver. broker-path-rules-receipt.json (evidence_class: synthetic, fixture
ledger only) records the passing promtool check rules/check config and
amtool check-config runs against configure.py's real rendered output.

tests/test_adaptive_paper_metrics.py builds its fixture ledger from
safety.Ledger's own serialisation for healthy, diverged and frozen scenarios,
fetches over a real loopback HTTP server, and proves the exporter never opens
a 0000-permission credentials canary. tests/test_observability_backends_alerts.py
checks the new templates and, when promtool/amtool are installed, renders and
validates them natively.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…adaptive paper runtime

Add promotion_gate.py: a pandera 0.33.1 / exchange_calendars 4.13.2 CLI, run
in its own isolated uv venv, that validates a data snapshot (unique
symbol/session, valid XNYS session, positive OHLC with high/low consistency,
non-negative volume, non-future observed_at) and writes a bounded
gate-result.json; fails closed with the raised exception's class on any
error. Two synthetic CSV fixtures (good/bad) exercise every named check with
retained gate outputs.

Extend runner.validate_preflight() with an additive, backward-compatible
mode="paper" path that requires a passing gate-result.json whose
input_sha256 matches the configured universe/bars input file (this runtime
has no live snapshot concept; Alpaca quotes are fetched fresh, not read from
a file), raising SafetyError with kind promotion_gate_missing/_failed/
_mismatch. Existing preflight/recover call sites (mode=None) are unaffected.
Wiring the --gate-result CLI flag and call-site arguments into runner.py's
main() is a follow-up step outside this change's owned validate_preflight/
credentials scope.

Harden runner.credentials() to fail closed (SafetyError
"credential_file_permissions", one-line remediation, contents never logged)
unless the --env-file is mode 0600, owned by the current uid, and outside
any Git worktree, before any line is parsed.

Record the credential-handling decision (native 0600 env file over OpenBao/
keychain/shell-exported env vars, with overturn conditions) in
docs/decisions/2026-09-22-broker-credential-handling.md.

Evidence: local_integration (promotion_gate.py run through its own venv on
both fixtures; new unittest coverage for the gate wiring and credential
checks). No broker call; no live credentials file touched.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

exchange_calendars was already pinned in catalogs/us-equities/data-research.json
(decision default, evidence_level native_proven, source dbe38b1f6887434bb...)
and catalogs/us-equities/architecture/trading-sources.json (source fetch commit
bbda29fed902374bdb75acab...) but absent from manifests/stack.json. Add it as
component id `exchange-calendars`, supporting profile, evidence_ids citing
native-historical-leverage-stress-20260919 (the only registered evidence.json
receipt that names exchange-calendars by function: its session_date_check
field records "Existing exchange-calendars XNYS schedule"). Cross-reference
the addition from the data-research.json/.md card.

Regenerated catalogs/us-equities/decision-index.json via the sanctioned
`scripts/catalog_decisions.py --write` (manifests/stack.json is one of its
base union sources) so scripts/validate_catalogs.py stays green, and
rehashed the touched files in manifests/evidence.json `files[]` via the
provided rehash helper.

Extended tests/test_stack_lifecycle.py and tests/test_ecosystem_manifest.py
to hold the new component to an explicit, narrow, documented allowlist
(PENDING_LIFECYCLE_AUDIT_COMPONENTS) instead of silently weakening the
existing full-coverage assertions, since blueprints/token-native-focus/
saturation-audit.json, catalogs/landscape/upstream-snapshot.json and
adoption/manifest.json recipe_map are outside this unit's owned paths and
need their own rows/entries added in a follow-up.

Known, reported acceptance gaps (see task handoff): scripts/validate.py
flags that native-historical-leverage-stress-20260919's own component_ids
does not yet list exchange-calendars (manifests/evidence.json is outside
owned paths); scripts/landscape.py needs a live-checked
catalogs/landscape/upstream-snapshot.json row (network + out of scope);
tests/test_adoption_contract.py needs an adoption/manifest.json recipe_map
entry (also out of scope). None of these were force-fixed by editing files
outside the owned-paths list.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

The JSON card's rationale, native_workflow comment and limitations
still described grype as source-reviewed/not executed even though
evidence_level was flipped to native_proven citing the scan receipt.
Update the prose to reflect what was actually run (grype db update,
grype db status, and the two sbom scans against the receipt), keeping
the disclosed scope limits (Python-metadata-only coverage, zero
matches is not a safety proof). Field set unchanged. Rehashed
manifests/evidence.json for the edited file.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

catalogs/us-equities/gates-20260922.json names 20 gates across the three rungs with owner, evidence class, receipt path and a machine-checkable flip condition; scripts/trading_gates.py --check validates the vocabulary, verifies every established gate's receipt and condition, lists flip candidates and computes rung readiness without any reviewer opinion. Established today: the offline equity replay and the Alpaca paper smoke. The parity gate is BLOCKED on the peer branch (29 checks, 4 failed on the two unsupported mappings); the wave-produced gates flip after integration.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

… and unsourced claims (pr4-cards)

Blockers:
- coverage.json/star-audit.json: nautechsystems/nautilus_trader's star record
  only listed catalog_entry_ids ['nautilustrader'], so validate_catalogs.py
  failed on the shared-repository nautilus-ibkr-adapter card. Both files now
  list ['nautilustrader', 'nautilus-ibkr-adapter'] for that repository, and
  coverage.json gains a beyond_stars entry for
  seathatflowsinourveins/native-agent-stack (adaptive-paper-alpaca-adapter),
  which was previously absent from both stars and beyond_stars.
  manifest.json counts.beyond_star_catalog_repositories: 105 -> 106.
  repository_entries=154/unique_catalog_repositories=148 were already correct
  (recomputed with alias canonicalization) and are unchanged.
- blueprints/token-native-focus/saturation-audit.json: counts.catalog_identities
  bumped 843 -> 844 to match decision-index.json's regenerated record count
  (the new adaptive-paper-alpaca-adapter repository adds one identity;
  nautilus-ibkr-adapter shares an existing repository record). catalog_card
  group 114 -> 115 and nonselected_catalog_identities 775 -> 776 so both
  tests/test_stack_lifecycle.py partition equations hold.

Major findings:
- engines-strategies.json/.md and reconciliations-20260922.json no longer
  publish the unsourced "SPY/LEAN parity gate G-a: 25 pass / 4 fail, confined
  to market_on_open_proxy and distributions_and_cash" numeric claim (grep found
  no receipt/gate file backing it). They now cite runtime-target.json's actual
  next_acceptance[retained-equity-replay].status: planned_not_executed and say
  explicitly that no retained receipt records pass/fail counts.
- Every "IBKR is the selected live-primary broker path" mention that cited
  runtime-target.json (engines-strategies.json rationale, agents-operations.json
  openbao limitations, agents-operations.md) is corrected: runtime-target.json
  broker_boundaries records only selected_path and
  local_broker_acceptance: not_established for IBKR, never the string
  "live-primary". The phrasing now attributes "live-primary" to this catalog's
  own decision and states what runtime-target.json actually records.

tests/test_catalogs.py: updated the parity-gate assertion to the sourced
language, and added regression coverage for the coverage/star-audit repository
partition (both nautilus_trader cards, the new beyond_stars entry, and the
beyond_star_catalog_repositories count) and for the live-primary attribution
fix.

manifests/evidence.json: rehashed the 9 changed catalog/test files via the
supplied rehash_evidence.py.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…x-round edit

The prior commit's rehash ran before the last edit to
tests/test_catalogs.py landed, so scripts/validate.py failed on a stale
sha256/byte count for that one file. Re-ran the supplied
rehash_evidence.py; only tests/test_catalogs.py changed.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

EcosystemServiceUnavailable now excludes job=adaptive-paper, since that
exporter is a separate process not started by install.py/configure.py and
would otherwise fire permanently whenever no paper trial is running.

metrics.py now always exports paper_ledger_readable (1/0, independent of
ledger content); a wrong --ledger path, a permissions problem, or a failed
read-only sqlite open previously left paper_order_state_divergence_total,
paper_ledger_frozen, and paper_request_budget_* silently absent with no
absent()-based rule able to target them. The new EquitiesLedgerUnreadable
alert guards this, and EquitiesPaperMetricsMissing's description no longer
claims to cover ledger-read failures it cannot detect.

EquitiesReconciliationFailed now also fires on
paper_reconciliation_status{result="needs_attention"} == 1, because
runner.py can end a trial at phase="finished"/status="needs_attention" (a
failed run recovered flat), which the phase-only paper_needs_attention
clause missed. The stuck-at-phase="starting" hard-kill case remains an
unguarded, already-documented gap pending an exportable last-success
timestamp.

Regression tests added to tests/test_adaptive_paper_metrics.py (unreadable
ledger, recovered-flat/needs_attention status) and
tests/test_observability_backends_alerts.py (new alert name, rule count
13->14, the exclusion clause, the added reconciliation clause). Receipt and
docs updated with fix-round promtool/amtool re-runs (14 rules found).

Co-Authored-By: Claude Fable 5.1 <[email protected]>

Resolves the fix-round findings for unit pr4-gate-credentials:

- promotion_gate.py: _summarize() no longer silently drops pandera failure
  identifiers outside the 11 hardcoded CHECK_NAMES (e.g. pandera's own
  `not_nullable` for a null OHLC cell). Unrecognized identifiers now surface
  as a synthetic `unmapped_failures` check with status "fail", so the gate
  fails closed instead of reporting "pass"/exit 0. Added a null-price-cell
  fixture and regression test that reproduces the exact scenario through the
  gate's own pandera 0.33.1 venv.

- runner.py main(): registers `--gate-result`/`--snapshot` and wires
  `mode="paper"` (with gate_result_path/snapshot_path) into the shared
  paper/recover call site, gated only for the `paper` command; `preflight`
  and `recover` are unaffected. Corrected the docstring's false claim that
  `--config` (JSON) could serve as the gated snapshot -- the gate only
  accepts parquet/csv/duckdb input, so that substitution always fails
  closed. `--snapshot` now names the actual bars/universe file that was run
  through promotion_gate.py; no such production ingest exists yet, which the
  README now states directly instead of implying `--config` works. Added
  MainCommandGateWiring regression tests driving runner.main() end to end
  for paper/recover/preflight.

- docs/decisions/2026-09-22-broker-credential-handling.md: narrowed Scope to
  what this unit actually changed (runner.py's credentials() only) and
  flagged market_research.py's unchanged, unenforced credentials() as an
  open follow-up instead of implying it is already covered.

Registered the two new fixture files in manifests/evidence.json and
rehashed.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…hness/adoption rows

Blocker: scripts/validate.py, scripts/landscape.py and
tests/test_adoption_contract.py were failing because exchange-calendars had
no genuinely dedicated evidence receipt, freshness-snapshot row or adoption
recipe_map entry.

Instead of retro-labeling native-historical-leverage-stress-20260919 (a LEAN
receipt with component_ids=["lean"]) to formally "cover" exchange-calendars,
add a new, honest, independently native_cli_e2e receipt
(exchange-calendars-native-check-20260922): 4.13.2 was pip-installed into an
isolated venv and its XNYS session_open/session_close for 2013-10-07 was
queried live, matching the values already recorded in
blueprints/us-equities/data/receipt.json; separate `gh api` calls confirm the
recorded source_pin and the 4.13.2 tag resolve to the identical commit. Point
manifests/stack.json's exchange-calendars.evidence_ids at this receipt only.

Add a real catalogs/landscape/upstream-snapshot.json row for
exchange-calendars using actual `gh api` GitHub metadata/commit checks (same
methodology as the other 68 rows), and update the file's summary counts.

Add exchange-calendars to adoption/manifest.json's recipe_map (pointing at
blueprints/us-equities/data/README.md, which documents its actual role) and
to the research-runtime profile's component_ids/recipe_paths, resolving
tests/test_adoption_contract.py.

Add a full blueprints/token-native-focus/saturation-audit.json lifecycle row
for exchange-calendars (install/use/persistence/restart/cleanup/recovery
stages, all evidence_refs resolving to real files) and update its counts
(selected_components 68->69, exclusive_identity_groups component_record
68->69 / catalog_card 114->113, nonselected_catalog_identities 775->774).

Restore tests/test_stack_lifecycle.py's full-coverage assertion
(audited_ids == stack_ids) instead of the PENDING_LIFECYCLE_AUDIT_COMPONENTS
allowlist/skip that the previous round added, and strengthen the dedicated
exchange-calendars test to assert the cited receipt's component_ids equals
["exchange-calendars"] exactly (guarding against future retro-labeling).

Rehashed all touched files in manifests/evidence.json `files[]` via the
provided rehash helper.

Verified (raw, from this worktree root):
- rtk proxy python3 scripts/validate.py -> status passed, 69 components
- rtk proxy python3 scripts/validate_catalogs.py -> passed
- rtk proxy python3 scripts/validate_foundation.py --root . --json -> ok true
- rtk proxy python3 scripts/landscape.py --root . -> status passed, selected_components 69
- rtk proxy python3 scripts/validate_convergence.py --all-recorded --root . --json -> valid true
- rtk proxy python3 -m unittest tests/test_stack_lifecycle.py tests/test_ecosystem_manifest.py tests/test_adoption_contract.py -> OK (55 tests)
- rtk proxy python3 -m unittest -> OK (1155 tests, skipped=204)

build_ecosystem.py --write was not run; docs/ecosystem/* and
catalogs/sota-convergence/manifest-20260922.json were not touched.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…ts supersede the pending note on the grype card)

… the saturation partition counts from the regenerated decision index

… with the gate ladder

Engine and broker cards reconciled (NautilusTrader 2.0.0rc5 default destination, LEAN frozen oracle, IBKR adapter and adaptive-paper Alpaca adapter cards), native syft/grype scan of the pinned rc5 runtime (zero matches on the 2026-09-22 database), broker-path alert rules with a read-only metrics exporter (synthetic), fail-closed snapshot promotion gate and credential-file enforcement, exchange_calendars in the stack manifest, reconciliations recorded, and the sim -> paper -> live ladder (20 gates, 7 established) with scripts/trading_gates.py --check. Saturation partition counts recomputed from the regenerated decision index; the cards test reconciled with the executed scan. Explorer regenerated and evidence rehashed; validators, gates check and 1,246 tests pass.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…ion index and saturation counts regenerated

The Nautilus card, its table row and the reconciliation notes now state the retained parity state (runtime-target next_acceptance reported_execution_blocked_review_incomplete; the comparison summary records a completed BLOCKED replay with four failed checks on the two unsupported mappings; not parity acceptance) instead of planned_not_executed; the ladder's parity note no longer cites a per-check total that is not retained on main; presence-only gates carry an explicit limitation and the checker rejects empty files for exists conditions (regression test). The five code findings (promotion gate fail-open paths, shallow gate-result validation, exporter path confinement and SQLite URI escaping) are fixed in a separate builder round.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…explorer regenerated

manifests/evidence.json: receipts unioned by id (exchange-calendars-native-check-20260922 from PR-4,
github-automation-newpc-20260922 from main), files unioned by path with every sha256/byte count
recomputed from the merged tree and the two publication_review fields preserved; docs/ecosystem/index.html
regenerated with scripts/build_ecosystem.py --write to its hash fixed point. validate.py 69/1740/139
passed; landscape, convergence (--all-recorded), build_verdicts --check, trading gates --check passed;
python3 -m unittest: Ran 1334, OK (skipped=204, system python).

Co-Authored-By: Claude Fable 5.1 <[email protected]>