Skip to content

Arm bounded overnight REST diagnostics with owned lifecycle and residual exits - #1081

Draft
seathatflowsinourveins wants to merge 11 commits into
mainfrom
foundation/overnight-arming-20261013
Draft

seathatflowsinourveins wants to merge 11 commits into
mainfrom
foundation/overnight-arming-20261013

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

Current PR head: 09a1426908b5dbb6ca0d8d4d621752bd82b131b2, tree 41b044269c420ba1ab43a5985d9ce411bb8626f7, one normal forward child of fddebeb9a79eed9e6dcb1cef19adff7c3b7ddd4d under the CC's 2026-10-14 01:39Z last-attempt ruling, 01:49Z conditional-correlation acceptance and 02:12Z all-owned-side liveness ruling. Broker UUID/request-marker correlation and retained refusal dedupe protect newer acknowledgements; every held cancel has bounded autonomous REST reconciliation and at most one lookup-confirmed retry per bound. Recovery joins owned cancellation work and its final lookup.

ONE final read wave: local review, Astra and the CC's Claude micro read this structural head. Any P1 in that wave, or no landing by 2026-10-14 06:30Z, parks #1081 and moves the cancel state machine to a design PR. Lane B stays on seq2 until a landed, armed head exists. Paper acceptance remains NOT_RUN. PR ready-for-review is set with FIX-PUSHED.

The prior task-only refusal correction was disproved by delayed/replayed refusal and is superseded by this structural correction. Known broker keys and consumed refusals survive local completion; missing, unchanged or ambiguous carriers stay held. HTTP acknowledgement followed by lookup failure also stays held. Native/worker and ownership controls are retained; further P2/P3 findings are Follow-ups.

Explicitly enable bounded BOATS REST overnight diagnostics with fresh quote and asset-carrier guards, client-side LIMIT exits, persisted carry deadlines and execution receipts. Entry refusals deny the order while other held positions keep running; owned reduce-only residual exits remain available in PRE/RTH/POST.

  • Base: 329985de1c8dccf0df7491373d0d9932eaa723d4; the mechanical rebase preserves all 103 non-listing path patches, compared with --unified=0 stable patch IDs against the old base ba6afd8d71893fdf190c4b137330c98fca15cc75 and local head d55c0484.
  • Lane: lane:trading.
  • Owned paths: adaptive-paper runtime and documentation, shared order-contract opt-in, focused tests, sanitized fixture evidence and hash bindings.
  • Paper acceptance: NOT_RUN. The separate mover overnight-awareness change is required before an overnight mover diagnostic can start.

The cross-family correction refreshes an exit's admitted market close at reservation and pending revalidation, including recovery on the same controller across 04:00. Entry carry/expiry timestamps and ledger cleanup budgets remain fixed. Adaptive exit cancellation/repricing first checks residual admission in every session. Successful asset reads deliver immediately even if the quote fails or returns 429; each symbol polls and reschedules independently.

Historical correction head before the mechanical main integration: f8d11a6eb76d67bd929c87974b9c73c1517e5c66, one fast-forward commit on 2658e7b8; tree 3554b76e56f50cb2e45287aa8a821d1ff67b062e. The privacy-squash rewrite was withdrawn under correction216. The later CC approval authorizes replay of the six commits onto current main with the full published-head lease; it preserves their content and does not purge historical digests.

Fixture output hashes now bind committed sanitized logs under evidence/artifacts/overnight-arming-20261013/. Historical public source preimages are published separately. The new publication regression rejects any receipt digest without a preimage in the published set and verifies every historical hash-map value against public bytes. The existing anchored gitleaks content-digest exception covers exactly the two validated historical map paths; tests keep credential-shaped keys and other paths detected.

Historical round-two verification and history scope

The controller consumes the real order_cancel_rejected trade-update envelope and releases the retry latch for a still-live entry. Overnight venue halts apply during OVERNIGHT; standing trading halts continue to protect daytime residual exits. Shared mover scan primitives use the calendar trade date and midnight-aware rule order; full mover configuration and command timing remain in the companion change.

Cancellation runs in tracked tasks with a five-second coroutine budget. This keeps cutoff and cleanup decisions moving while the broker request waits. Astra's real-SDK probe showed that the original timeout abandoned synchronous SDK workers; the original coroutine fixtures did not establish worker ownership through transport shutdown. The forward correction below retains and joins actual work; it establishes no broker latency guarantee.

  • Exact448 base: seven functional tests fail15 assertions with zero errors. Eight removal mutants are killed by11 assertions, zero errors.
  • Final native package-mode related suite:726 PASS, zero skips,313.415s.
  • Same CI-like modules:726 PASS,220 native-only skips,97.362s.
  • The first native726 run had two assertions in the existing halt-delivery fixture expecting synchronous cancellation. Its immediate synthetic callback now gets one event-loop turn; the failed run and clean repeat are separately published.
  • Final blind-checkout/four-hook/source-map/publication shard:57 PASS, zero skips,94.821s. Full integrity validator:71 components,11951 hashed files,240 receipts.
  • New fixture receipt: blueprints/us-equities/adaptive-paper/overnight-review-round2-validation.json. Sanitized output hashes bind published bytes; six exact public historical source blobs preserve older execution-input hashes. Derived registry hashes have their own listing checks.
  • The generic seven-character-prefix regression derives synthetic values. The six historical values are supplied privately for the actual tree/body audit, with no public blacklist.

Correction216 withdrew the proposed privacy rewrite. Historical SHAs remain viewable; no purge is claimed. The later approved mechanical main rebase preserves all non-listing patches. The historical round-two receipt binds parent448 and its own source snapshots; current integration and forward-correction results are recorded separately below. Both listing rows and the approved mover import union are validated on the integrated tree.

SOTA sources

  • alpaca-py v0.44.0, cc4cb3b7ba50ae250e621983c2779047fb16bb28: alpaca/data/historical/stock.py:163-182, StockLatestQuoteRequest, TradingClient.get_asset, Asset.attributes. Its supported BOATS REST API is used; its stock stream still refuses BOATS.
  • NautilusTrader 2.0.0rc5, 1b0a49d2792a9432a3aca3fcb617ce7a630d905e: python/nautilus_trader/live/clients.py:451-455 queues denial; crates/live/src/python/node.rs:175-183 stops the node; crates/risk/src/engine/mod.rs:727-771,2305-2328 defines eligible reducing submissions; crates/model/src/python/orders/limit.rs:260-264 exposes is_reduce_only.
  • exchange-calendars 4.13.2, dbe38b1f6887434bbdd1a7d2df6ff8f1742a048a: XNYS session APIs used by existing sessions.session_at and overnight_bounds.
  • Alpaca Orders and 24/5 Trading: extended-hours simple LIMIT DAY/GTC and overnight asset attributes. Current production factories emit DAY; genuine broker acceptance remains pending.
  • Existing repository scripts/host_receipts.py:sanitize, 2026-09-24 community sweep: effort-guard SessionStart notice, working plugin-install commands, dated dispositions #161 published-byte rule, and the line-anchored .gitleaks.toml source-map exception: adopted for sanitized public preimages and narrowly scoped historical-map validation.

Evidence-class table

Claim Evidence class Evidence
Same-controller OVERNIGHT-to-PRE reservation, pending revalidation and automatic recovery synthetic test_adaptive_paper_overnight_transition.py; exact parent red, fix green
Halt refresh survives quote failure/429; accepted entry cancels while resting held exit stays synthetic Same module, real carrier/ledger path
Healthy symbol delivers and repolls while another request waits synthetic Production poller and source fixtures
Production mover/native adaptive exits carry reduce-only metadata and each F1-F4 layer refuses independently synthetic test_adaptive_paper_overnight_guard_pins.py, native emitter plus CI-visible pins and mutants
Each published fixture digest resolves to sanitized published bytes local integration test_overnight_published_preimages.py, repository digest/privacy and scanner tests
SDK, calendar and engine semantics source_review Pins above and decision record
BOATS/SIP entitlement, live halt delivery, actual order/fill/cadence behavior NOT_RUN Owning-lane readiness and retained acceptance required

Local commands run

All runs completed in serialized idle scopes; native fixture runs held the canonical runtime read lock.

  • Exact fix parent 2658e7b8: 10 functional/publication tests, 18 assertion failures, zero errors. The required same-controller, recovery and halt cases fail before; their fixtures never repair the controller close.
  • Related native suite: 547 passed, no skips on the repeat. The initial run had one short-trial timing failure in IntegratedRunner.test_native_adaptive_policy_reaches_ledger_and_flat_reconciliation; that case passed alone at the fix and exact 9cb7c0d3. The failed run and isolated runs are separately published; no code changed for it.
  • Final native split-boundary/publication shard: 11 passed, no skips.
  • Final CI-like package-mode related shard: 138 run, 2 native-only skips, exit 0. Base CPython 3.12.3, env -i, scratch USER-site hash-locked calendar dependencies, no Nautilus/alpaca.
  • New transition mutation plans: 7 killed by 16 assertion failures, zero errors/survivors. F1-F4's 19 plans remain killed, including pair/triple transport deletions and the production mover emitter; original 37 plans remain killed.
  • Repository published-digest privacy plus publication tests: 18 passed with host-name guard enabled; final publication/registry guard 19 passed.
  • Native gitleaks context-restriction regression class: 31 passed, including exact historical-map path/key constraints.
  • Full python3 scripts/validate.py: passed, 71 components, 11,935 hashed files, 240 receipts. Redacted gitleaks scan of the committed PR range ba6afd8d..f8d11a6: exit 0, no leaks. The committed repository privacy/publication modules passed 18 tests with host-name guarding enabled. All 54 changed evidence bindings and 12 fix input hashes were recomputed and verified.

Records: overnight-arming-validation.json, overnight-guard-pins-validation.json and overnight-review-fix-validation.json under adaptive-paper. Every output digest names committed sanitized bytes. Historical receipts retain their input scope through published historical source preimages.

Decision record

docs/decisions/2026-10-13-overnight-rest-arming.md records the supported REST adoption and the reproduced same-controller, lost-halt and private-preimage corrections. README-overnight.md describes the opt-in, independent polling, residual session checks and acceptance boundaries.

The historical pre-push PASS records are arming-prepush-claude-read-staged-e7d9ccb4.md, overnight-arming-prepush-claude-read-restaged-a9ad1d71.md and overnight-arming-prepush-claude-read-rebased-9cb7c0d3.md in coordination/trading-cc/reads. They authorized the original push. Their overall PASS verdicts were subsequently withdrawn by arming-1081-claude-correction-2658e7b8.md after the reproduced P1s; this draft awaits the new exact-head reads and gate.

Host evidence

No host platform-status change or broker acceptance is contributed.

Checklist

  • No workflow change, credential publication, new paid service or real broker action.

  • Peer tests/worktrees and protected trading inputs preserved.

  • Original release pushed once; subsequent fixes use fast-forward commits, with no history rewrite or hook bypass.

  • Paper acceptance remains NOT_RUN.

  • Final exact-head trading-cc gate and both review families.

  • All hosted shards at the corrected head assessed before landing.

  • Round two: Alpaca's official streaming source at07f9c20b, installed alpaca-py0.44 TradeUpdate.event, and Python3.12 asyncio.timeout; dates, current-doc HTTP403 and fixture limits are recorded in the decision.

Receipt completion (CC05:01 ruling): current head 9d43e306e9bf8f43c758717a02898b408fcc30a7, tree bac49c4661d459244c8d40f8c54329402499a73a, is rebased onto main 67469fb6ba5476459d3d8dac9d11bc62dd1c1f4b. The only new completion commit publishes the four lost exact Git preimages plus the auto-merged source-policy and publication-test registration dependencies, and adds a new integrated validation record, SHA256 b2f0713e417d39a647c0b06c6863bcdba22b808db458e4a59c24ce5f964cda6b. Existing sealed receipts retain their historical bindings and exact bytes. Trading source remains byte-identical to reviewed 02e19f10. On the exact committed head before push: publication tests 4/4, ten native modules 474/474 (all 14 cancellation tests ran; zero skips), validator rc 0, merge-tree against the stated main rc 0. The failed namespace-only fixture calibration is retained separately and is not a passing result. Current head is frozen for CI, the co-op native receipt and the Astra/CC Opus completion wave; landing and broker acceptance remain pending.

Follow-ups

  • FIRST: trading-cc P2 — hold_422_nonterminal_cancel (CC04:53 Follow-up). At reviewed 09a14269: transport.py:1966-1967 and :2140-2143; the immediate owner retry is at runner.py:1135-1139 and :1151-1167. A DELETE answered 422 followed by a nonterminal GET releases pending, so another DELETE/GET can run on the next tick (about 12 ms). The fake probe reproduces the same behavior at fddebeb9a; this is pre-existing and the CC adjudicated it P2, not a landing blocker. Named fix: hold a 422 whose follow-up GET is nonterminal and let the bounded reconciliation handle it. Fail-first regression: assert bounded DELETE and GET counts across repeated ticks and the reconciliation interval, including pending_cancel/accepted and terminal release; extend the existing test_adaptive_paper_transport.py:779-787 raise-only case rather than accepting only its exception assertion. Owner/trigger: ns-seeds' own small PR after the quiet box lifts. No code change or new head in this read wave.
  • trading-cc P3-1 — final DELETE-sink stop/terminal check. Reviewed transport.py:2111: a stop or terminal stream event during the budget wait can still allow one idempotent DELETE after the last stopping check (F1). Trigger: cancellation lifecycle design work; check stop/terminal admission at the actual DELETE sink and test the post-budget race.
  • trading-cc P3-2 — cancellation return and logging contract. cancel() may return a joined reconciliation's lookup result, None after a 404, or a prior result without sending DELETE; the adapter may neither observe nor fail, while the runner logs overnight_entry_cancel. Trigger: cancellation interface/lifecycle design work; distinguish a sent cancel, a joined lookup and no new wire action, and test adapter observation plus truthful controller logging.
  • trading-cc P3-3 — pending-latch transitions during retry/stop. Reviewed transport.py:2078 and :2055-2056: retry_authorized is temporarily resolved before its completion callback, and stopping can discard pending for a broker-live order (F8a/F8b). A fresh recovery port prevents the exposure path in the reviewed cases. Trigger: lifecycle design work; define and test the logical pending invariant through authorized retry, completion and shutdown handover.
  • trading-cc P3-4 — REST confirmation versus cached stream state. Reviewed transport.py:2139: confirmed() can bind from the cached stream view when the REST row is stale. Trigger: cancellation-correlation design work; test stale REST/newer stream provenance and keep broker-key confirmation attribution explicit.
  • trading-cc P3-5 — cancellation counter semantics. held_unattributed_refusals includes refusals attributed later and excludes refusals removed by stale filtering (F11); lookup_confirmed_retries increments when the wire starts. Trigger: counter-validation follow-up; document ingestion/attribution/stale-drop/wire-start meanings and assert all four paths, extending the existing counter item below.
  • trading-cc P3-6 — stop caller cancellation and self-join contract. Reviewed transport.py:2247-2252 swallows caller cancellation without re-raising; the :2241 comment that it cancels only the caller's wait does not describe that behavior (F9a). A caller from _tasks or _cancel_operations awaiting port.stop() could hang (F9b; the base also fails, and no such caller exists at the reviewed head). Trigger: lifecycle design work; reconcile the public stop contract/comment with owned teardown and explicitly test cancellation and prohibited/self-join callers.
  • trading-cc P3-7 — task retention and autonomous-retry scope. Retry tasks remain in _tasks until stop; the monitor arms for every held owned cancel, including RTH transport use. The reviewer did not read the CC02:12 all-owned-side liveness ruling. Trigger: lifecycle design work; reap completed tracked retry tasks and retain an explicit scope/policy record with RTH BUY/SELL/recovery coverage. Do not present the synthetic regressions as measured broker acceptance.

These eight trading-cc dispositions follow its PASS/no-P1 read of 09a14269, sealed record coordination/trading-cc/reads/nas1081-claude-read-09a14269.md, SHA256 b9843d494bbd742a533a4d41b82183c798cde678a05acd3096762f15f7f7e632. The review's probe limitations remain: copied controller logic/plain-task recovery stand-ins are separately attributed, and actual broker marker/event/cancellation behavior is unmeasured. The CC04:53 ruling leaves 02e19f10 under read; these are body-only dispositions.

  • Astra fix5 P2 — terminal REST observations and native order state. At the reviewed 09a14269 tree: runner.py:1147, transport.py:1853 and transport.py:1687. A background cancel lookup reaches the controller before the native execution observer. Trigger: cancellation lifecycle design work; deliver idempotent terminal REST observations to the native order-state path while preserving execution/fill deduplication, with a delayed-trade_updates regression. This remains an open follow-up under the CC's severity ruling.
  • Astra fix5 P2 — explicit integration with Separate extended-hours paper startup readiness from order freshness #1097. Reviewed 09a14269 locations: transport.py:1325 (constructor), :1718 (quote age), runner.py:2671 (preflight), plus both listing files. Trigger/disposition: the landing rebase onto 59899aaf prepared 02e19f10 with four preserve-both code hunks; main's readiness reservation and benchmark maximum remain alongside overnight recovery feed, kwargs and BOATS minimum age. Both registries were regenerated without dropping main rows; cancellation tests 14/14, readiness tests 18/18, no skips, validator rc 0. The new resolution wave must verify this integration before landing. Per-hunk record: pr1081-landing-resolution-59899aaf.md, SHA256 0dbd0989239ce86cacca8675db454ef0bf3c7279a1f8116c11a41688eec9e997.
  • Astra fix5 P2 — inherited catalog-freshness failure. Reviewed locations: tests/test_catalog_freshness_propose.py:830, catalogs/foundation/model-currency.json:127, scripts/validate.py:248. The historical failure repeats and is not classified as flaky. Trigger/disposition: carry main's reviewed Restore model-currency validation with sourced Fable and Astra landscape checks #1101 correction (04926b5d) through landing and rerun current validation. This correction is present in 02e19f10's main base; python3 scripts/validate.py returned rc 0 on that exact prepared tree. Historical sealed passes alone do not qualify a later head.
  • Astra fix5 P3 — missing primary-source captures. Reviewed locations: docs/decisions/2026-10-13-overnight-rest-arming.md:335 and :342, overnight-cancel-identity-validation.json#/sources, and the body's cancellation citations. Websocket/Orders page locators and the older owner-free handoff, vendor/24-5, halt-policy, UTP and historical-review references lack the corresponding captured bytes in the supplied read packet. Trigger: source-publication follow-up; add sanitized primary captures or pinned blobs with matching hashes and locators. Keep dependent claims unverified until captured; the absence of a capture does not establish that a vendor claim is false.
  • Astra fix5 P3 — CPython shutdown citation endpoint. docs/decisions/2026-10-13-overnight-rest-arming.md:229 cites python/cpython@f6650f9ad73359051f3e558c2431a109bc016664:Lib/concurrent/futures/thread.py:219-241; the pinned file has 239 lines and 219-238 supports the claim. Trigger: citation-maintenance follow-up; change the endpoint to 238. No test-enforced locator failure was reproduced; this body-only disposition changes no source at the read head.
  • Astra fix5 P3 — exceptional cleanup ownership and locator qualification. Reviewed transport.py:2294 raises on a live stream thread before subsequent HTTP-session closes; this existing stream-leak branch is distinct from the tested cancellation-worker join. Also retain qualifications on historical/pinned references and current maintenance comments such as tests/test_adaptive_paper_metrics.py:149. Trigger: lifecycle design work; make exceptional cleanup ownership explicit and test it. Update current-code locators when they move while retaining historical pins. Successful cancel regressions do not establish universal cleanup success.

Rebase note: the CC authorized one leased landing rebase from 09a14269 to the prepared 02e19f10 on 59899aaf. The four code resolutions preserve #1097's request reservations/readiness and the overnight behavior together; the new head requires its own resolution wave. Astra fix5 is a final unconditional ACK of 09a14269, not a broker-acceptance, arming or landing authorization. Its sealed verdict SHA256 is 15ef329785a56d15b1c7c9dbb8a900d6229b947cc3449ece1a401116175022e7.

  • CC micro P2 — refusal counter attribution: blueprints/us-equities/adaptive-paper/transport.py:954-959, compared with:969-973. held_unattributed_refusals counts a refusal at ingestion that can be attributed later; no current test asserts counters on that later-attribution path. Trigger: the cancellation-state design PR or the next counter-validation change; distinguish permanently unattributed refusals from temporarily unresolved attribution and add that counter assertion.
  • CC micro P2 — pacing after a definitive DELETE refusal: blueprints/us-equities/adaptive-paper/runner.py:1135-1139. An explicit4xx DELETE refusal releases the latch and the runner can re-cancel on every tick. Trigger: the cancellation-state retry design or the first observed venue refusal; back off according to the venue's wait, following the bounded transient policy in#1091, and retain a test that forbids per-tick retries during that wait.
  • Named first paper stage check: cancellation request carrier (first_stage_cancel_carrier). At the first owning-lane paper stage, measure whether cancel_requested_at is populated and advances per DELETE, and exactly what a late order_cancel_rejected observation carries. Retain sanitized request/response/event observations and counter values. This check is still unmeasured against the broker; nullable example fields and synthetic fixtures do not establish venue behavior.
  • Session-boundary races: trigger on an owning-lane boundary refusal or before widening the run; measure owner/dispatch/wire clock crossings and retain the reproducer.
  • GTC and recovery payloads: trigger before the first executable GTC diagnostic; retain broker cancel/expiry/recovery acceptance. Current factories remain DAY.
  • Non-overnight lifecycle events: trigger before another lane adopts halted-entry cancellation or trail-high events; compare its order/event contract.
  • Event-memory growth: trigger before extending bounded duration or reaching the host memory budget; measure volume and adopt bounded telemetry.
  • Clock skew: trigger on negative/inconsistent broker latency or before using it in a gate; separate event and observation clocks.
  • Sticky stale transport health: trigger when fresh REST observations follow a stale freeze; verify reconciliation recovery before relaxing fail-closed state.
  • Inherited share-cap citation: trigger on the next share-cap doc edit; resolve the current safety.py bound and update the old locator.
  • Inherited buy-submissions citation: trigger on the next recovery doc edit; replace old recovery.py:168 locators with the current literal's line.
  • Capture availability: trigger before cross-host reproduction or widening qualification; publish missing vendor captures and review-packet inputs. Sanitized fixture logs and historical public source preimages are included here.
  • Brief attribution/locator: trigger before publishing the coordination brief outside its current scope; derive a cited sanitized technical record.
  • Session-specific extended-hours flag: trigger before a caller constructs its own reduce-only exit payload; current native dispatch derives the flag from the active session.
  • Authoritative halt delivery outside OVERNIGHT: trigger before real PRE/RTH/POST residual acceptance; verify a maintained live halt source. A fixture standing halt does not establish live coverage.
  • Exit-refusal node-stop policy: trigger on an owning-lane transient exit refusal terminating a run; revisit with the CC. Current behavior retains the explicit residual handoff rule.
  • Astra P3-11 provenance captures: trigger before wider qualification or the next correction-record edit; publish sanitized vendor-page/PDF captures and pin the adopted mover-child primitive paths/lines. Executed SDK-envelope tests remain separate from unavailable vendor-page capture verification.
  • Astra P3-12 main import conflict: resolved by the approved preserve-both union in the mechanical rebase onto329985de; trigger on a subsequent main integration conflict and require a fresh merged-tree test.
  • Astra P3-13 lifecycle/locators: trigger before widening DAY/GTC recovery, session halt delivery, retry clock retention or acknowledgement-time exit behavior; retain historical pin semantics and refresh current unpinned comments on their next documentation edit.
  • CPython source-range follow-up: trigger on the next source-citation maintenance pass; correct the historical decision/receipt endpoint thread.py:219-241 to219-238. The supported shutdown implementation and joining behavior are unchanged. Hostname hygiene is removed as a finding under the owner's22:25Z ruling.
  • Inherited synthetic mover entry timing: trigger before relying on the 2s fixture entry window under concurrent native-suite load. The unchanged snapshot-latency test reproduced completed_no_signals in1/3 isolated runs at exact8f64 and1/3 at the worker-fix tree, with no adapter errors; its MoverSimulatedPort does not execute the changed SDK transport. Retain both failing measurements and the full native failed run alongside the repeat.
  • trading-cc P2-1 halt-delivery wiring: trigger before wider overnight qualification; pin the complete production poller-to-controller delivery chain with the surviving M13c removal control. Current behavioral fixture tests do not claim that specific wiring is pinned.
  • trading-cc P2-2 recovery loop refresh: trigger before accepting a run that recovers across04:00; pin the actual recovery loop callsite against M5, beyond the existing same-controller tests.
  • trading-cc P2-3 last-second recovery: trigger before extending operation to a03:59:59 residual; resolve and measure that recovery window. The bounded diagnostic's earlier flat deadline remains its operational limit.
  • trading-cc P2-4 runner cleanup sites: trigger before relying on the native/mover wrapper-join callsites as independent safety guards; pin both runner shutdown paths and the task-map guard. This correction directly tests transport ownership, controller latch retention and SDK shutdown joining.
  • Inherited achieved-leverage fixture: trigger before using short real-time native warmup windows under suite load; the final native787 run fails its zero-exposure assertion, the same exact8f64 production fixture fails1/3 isolated runs, and the combined sources pass3/3. Preserve the native rc1 result and both paired measurements.
  • trading-cc P3 qualifications: trigger on the next evidence/citation edit or before widening operation; maintain declared receipt registry-input scope, separate fixture standing halts from actual daytime BOATS coverage, retain outcome/scheduling clock distinctions and retry-budget limits, publish missing provenance captures, and keep real private-prefix audits separate from synthetic guard tests. The actual5s SDK test now pins the timeout; controller/strategy mark ownership is addressed in this correction.

The separate mover-awareness change and readiness must land before overnight-diag-1. No strategy claim, live operation or model order authority is introduced.

HTTP422 cancellation blocker and main integration

A refused DELETE followed by a still-live broker lookup raises TransportError("cancellation_unresolved"); unknown identity and missing finality also raise their explicit transport failures. Those completed failure outcomes release the pending mark for a retry. The previous5b34905 callback incorrectly released it after HTTP204 plus a live lookup; that preservation claim is superseded by this correction. HTTP204 plus accepted, new or pending_cancel now retains all pending marks until a terminal stream/later lookup, while terminal HTTP404/422 observations remain resolved data.

The production-path synthetic-I/O regression is OvernightCancelRefusal.test_422_live_lookup_releases_latch_retries_and_stops_entry_at_cutoff. It fails at the exact published ea0a51a6 baseline (one assertion failure, zero errors), then passes with the blocker fix. The pre-rebase native test plus transport module passed 132 tests. Its receipt and public historical input snapshots retain their original pre-rebase bindings; post-rebase checks below validate the integrated tree separately.

Rebase note: the only manual code conflict was the adjacent import block in mover.py. The approved union preserves main's risk_ceiling_kwargs import and this PR's SessionKind / session_at imports. No function body was manually merged. All 103 non-listing per-path stable patch IDs are equal, including mover.py. Both listings were regenerated from main plus the original PR rows, followed by scripts/evidence_manifest.py --write; all main and PR keys, evidence rows and receipt IDs are retained.

Post-rebase checks: full scripts/validate.py rc0 (71 components, 12001 hashed files, 240 receipts); CI-like related suite 778 tests OK with 220 native-only skips; hook/blind-checkout runner 45 tests OK with no skips; publication/source-map modules 12 tests OK; pinned mover import from its source directory OK. The committed six-prefix audit resolves all six private inputs and finds zero matching locations; all 36 published sanitized excerpts have public preimages. Gitleaks scans the six rebased commits with rc0 and zero leaks. All jobs run in idle CPU class; all broker I/O is synthetic.

The post-rebase native related suite passes all 778 tests with no skips in 223.212s under runtime_read_lock(). Its two mover modules pass all 107 tests; the CI-like mover modules pass 76 tests and skip 31 requiring the pinned native runtime. The CI-like complete related run takes 77.085s. The four hook tests plus the complete blind-checkout module pass 45 tests in 83.968s. git merge-tree --write-tree 329985de HEAD exits 0 and yields this exact tree.

Forward P1 exception: actual SDK cancellation ownership

Under the CC's21:13Z approval, one normal forward correction on published8f64 retains a shielded per-order transport operation and its actual concurrent SDK future. The five-second owner timeout releases only its wait. Retries await the same operation; a late completion forwards its final observation. Shutdown cancels operations waiting for the operation lock and not-started futures, joins started calls and reconciliation, and joins the supported ThreadPoolExecutor before closing the SDK HTTP session. The copied context retains the existing per-order budget identity. The SDK implementation is unchanged.

Pinned CPython3.12.3 Future.cancel() explicitly refuses to cancel running work (python/cpython@f6650f9a, Lib/concurrent/futures/_base.py:364-372), and its supported executor shutdown cancels queued work and joins worker threads (Lib/concurrent/futures/thread.py:219-241). The installed alpaca-py0.44.0 cancellation remains alpacahq/alpaca-py@cc4cb3b7, alpaca/trading/client.py:217.

The final real-SDK module tests.test_adaptive_paper_cancel_worker_ownership covers actual5s HTTP timeout, blocked budget, late completion, queued shutdown, and halt-clear/order-timeout retry after a real422/live lookup. Five cases fail four assertions with zero errors/skips against exact8f64 production modules, including the money-path assertion of zero strategy retries before flatten; all five pass at this head in9.822s. Four CI-visible production-method controls fail four assertions at the base and pass at the head. They cover both adaptive/mover mirrors and preservation of unrelated engine-cancel marks. Five removed guards fail7 assertions total with zero errors: shielding, queued-work cancellation, controller live ownership, adaptive mark release, mover mark release.

Controller pending state remains set through a running SDK operation and, following its HTTP204 acknowledgement, through a nonterminal broker snapshot. Confirmed terminal state or definitive refusal/failure releases it; the next strategy pass clears only its mirrored background mark. Finishing a successful DELETE await alone does not authorize another DELETE. The historical q9 probe reports zero engine cancels before flatten at exact8f64 in both failure modes, and one at5b34905 in both. Its strategy is a protocol owner, not a running rc5 node; the separate worker tests exercise the installed SDK/transport against synthetic HTTP.

Final CI-like related suite:787 tests OK,225 native-only skips,111.087s. Final native suite:787 tests, one inherited achieved-leverage fixture failure, zero errors/skips,260.185s; it is rc1, not a passing full-native claim. The identical zero-exposure assertion reproduces1/3 at exact8f64 production and0/3 at this head in paired isolated runs. All nine cancellation regressions pass. Earlier worker-only782 failures and their exact inputs are preserved separately and do not stand in for combined-source validation.

New receipt: blueprints/us-equities/adaptive-paper/overnight-cancel-worker-validation.json; preserved prior-stage receipt: evidence/artifacts/overnight-arming-20261013/review-cancel-worker/worker-only/validation.json. The seven-receipt/source-map shard passes12 tests. Final full validator rc0:71 components,12051 hashed files,240 receipts. Four hook tests plus complete blind-checkout:45 PASS/no skips,149.477s. All62 published sanitized excerpts have public preimages; six private-prefix inputs resolve with zero tree matches. The original422 regression waits for its tracked cancellation task before checking the failure latch, preserving its assertion and production-path behavior. No broker or actual credential is used, and no SDK is replaced or forked.

Narrow acknowledgement correction

Runtime delta: one completion callback,7 lines added/3 removed. The new test CancelAcknowledgement.test_three_ticks_keep_acknowledged_live_cancel_pending_until_terminal executes the reviewer's three-tick scenario using the real pinned SDK, GuardedSession, Controller, ledger and request budget. It covers accepted, new, pending_cancel, with terminal stream and later-lookup controls for each. At exact5b34905 all six subcases fail [1,2,3] versus [1,1,1] DELETEs, with zero errors. At this head all six send one DELETE, reserve one cancellation budget slot and hold controller/adaptive/mover marks through all three ticks; terminal confirmation releases them without another DELETE. Removing the guard fails the same six assertions.

Alpaca's primary Orders page L1560 states that canceled orders receive no further updates; L1563 defines pending_cancel as “The order is waiting to be canceled.” The pinned OrderStatus enum distinguishes these statuses; installed enum bytes match that mirror.

Focused native integration218/218 PASS/no skips,39.807s; CI-like218 OK/93 native-only skips,50.834s, covering transport, recovery, prior worker/shutdown/422/strand tests and the new regression. Eight-receipt/source-map12 PASS; full validator rc0 (71 components,12063 files,240 receipts); hook/blind-checkout45 PASS/no skips,88.755s. All67 sanitized excerpts have public preimages and exact prior5b source snapshots preserve earlier receipt bindings. New receipt: blueprints/us-equities/adaptive-paper/overnight-cancel-ack-validation.json. This focused run is not represented as a new full adaptive-paper suite or live-broker acceptance.

Historical refusal-provenance correction (superseded)

Historical runtime delta at fddebeb: transport10 lines added/1 removed. Current-task attribution was disproved by Astra's delayed/replayed refusal probe; it allowed an older refusal to release the retry's acknowledgement. The source and validation below retain that stage's measured results, and the structural cancellation correction supersedes its preservation claim.

CancelAcknowledgement.test_rejection_during_lookup_releases_pending_and_allows_one_retry delivers the rejection on the actual owner loop while the pinned SDK getter/follow-up lookup is live. Six accepted/new/pending_cancel × terminal-route subcases fail at exact35df (DELETE counts1/1/1, marks stuck), then pass with exactly one retry (counts1/2/2, marks released after refusal and retained on the retry's acknowledgement). The original no-refusal test stays1/1/1. Both tests pass7.321s; removing refusal consumption fails the same six assertions with zero errors. Native focused219/219 PASS/no skips32.251s; CI-like219 OK/94 native-only skips29.747s.

Primary Orders page L1564 documents that pending_replace “will reject cancel request while in this state.” At alpaca-pycc4cb3b7, alpaca/trading/enums.py:321-333 defines TradeEvent names, while alpaca/trading/models.py:640 accepts Union[TradeEvent, str]; alpaca/trading/stream.py:115-139 dispatches/casts trade updates. That pin has no order_cancel_rejected enum member; the string-compatible path carries the event. These primary sources were read first-hand.

Nine-receipt/source-map12 PASS; historical sealed full validator rc0:71 components/12075 files/240 receipts; hook/blind-checkout45 PASS/no skips93.007s. All72 sanitized log excerpts resolve to public preimages; exact35df transport/doc/publication/ack-test snapshots preserve earlier input hashes. Receipt: blueprints/us-equities/adaptive-paper/overnight-cancel-provenance-validation.json. These checks predate midnight's model-currency freshness incident and do not establish current readiness.

Structural cancellation correction: final attempt

The CC's 2026-10-14 01:39Z ruling and 01:49Z conditional-correlation acceptance authorize one structural forward commit on fddebeb. A copied ContextVar binds each local attempt to its exact SDK DELETE at the HTTP seam after its budget wait. HTTP acknowledgements and definitive refusals stay with that identity. A 204 followed by a failed lookup keeps holding; the failure cannot authorize another DELETE.

Stream refusal correlation is conditional on a broker key: (order UUID, cancel_requested_at), confirmed by that operation's follow-up response and strictly advanced from its pre-operation broker marker. A prior operation with unknown identity makes later markers ambiguous. Keys and refusal observations remain deduplicated after local completion. updated_at, local receive time and the current task never infer a missing request identity. An older or unattributed refusal cannot release a newer acknowledgement, including after its task has completed. The transport latch covers direct adapter callers too.

The primary Websocket Streaming page, read first-hand, documents pending_cancel at L158 and order_cancel_rejected at L163; L178 includes only a nullable cancel_requested_at example field and documents no per-DELETE id. The pinned alpaca-py Order model does not declare it; selected raw_data=True preserves raw broker carriers when present. The Orders page L1560, L1563 and L1564 supplies the terminal/pending/refusal definitions. No upstream guarantee of universal request correlation is claimed.

A transport-owned monitor gives every held cancellation an independent REST lookup on a fixed five-second bound without requiring owner ticks. The CC's 02:12Z ruling includes held SELL and recovery cancellations. Terminal results release it; pending_cancel retains it; another nonterminal result authorizes exactly one lookup-confirmed DELETE per bound through the ordinary SDK/budget/operation registry. Run-end stream-health and recovery receipt fields expose held_unattributed_refusals and lookup_confirmed_retries, including nonzero values. Recovery joins the supported in-flight SDK budget wait, final lookup and teardown; its unrelated ten-second stop timer is removed. Caller cancellation cannot release the account while the transport still owns a worker.

Five core regression tests fail at exact fddebeb: replay/delayed refusal, acknowledgement plus failed lookup, hold reconciliation, cutoff flatten, and recovery timeout (15 assertion failures, zero errors). The two explicit SELL/recovery cases add two baseline assertion failures and zero errors. Both relocated Astra modes fail six assertions at that baseline. Replay now sends1/1/1 DELETEs; delayed mode sends1/2/2 in total (the initial refused A plus exactly one acknowledged B, no C). The permanent test additionally covers identified/missing keys and delivery during lookup/after completion. The earlier no-refusal and correlated-refusal controls remain green; the latter now supplies the documented broker marker. Removing correlation fails three assertions, removing recovery ownership joins fails one, and removing the autonomous monitor fails both SELL/recovery cases; each control has zero errors.

The held-at-cutoff BUY test drives actual pinned SDK cancellation, Controller and durable accounting, then the production deterministic recovery exit loop with the repository's synthetic fill/position oracle. Two additional SELL/recovery tests use the actual constructor, SDK DELETE/GET/POST, GuardedSession, Controller, ledger, recovery and reconciliation with synthetic HTTP. Held SELL cancellation permits no second DELETE inside the bound, retries once after a live lookup and reaches flat with one owned reduce-only SELL; held recovery cancellation releases on a terminal lookup and reaches flat with just one DELETE. Recovery reports its nonzero held counter. These paths are not measured against the broker, and these specific probes do not run rc5 nodes. The ownership test crosses the actual10.2-second boundary with the SDK worker still waiting for its supported budget callback and verifies pending ownership, final lookup and close ordering. All new session clocks are fixed; real elapsed time is used only for async waits/watchdogs. The new receipt is blueprints/us-equities/adaptive-paper/overnight-cancel-identity-validation.json; exact prior fddeb input snapshots preserve historical evidence bindings.

The first broad all-side native run found one test-fixture deadline failure: the0.5s confirmation wait expired before the virtual5s advancement under load. A controlled0.7s fixture delay reproduces cancellation_not_confirmed with recovery already stopped; increasing only fixture budgets passes. Confirmation now6s, cleanup20s and watchdog10s; no runtime change was made for that calibration. The failed diagnostic remains published and is not final-tree acceptance.

Validation and the exact published head are recorded at FIX-PUSHED after sealing. This head gets ONE local/Astra/Claude read wave. Any P1 in that wave, or no landing by 2026-10-14 06:30Z, parks #1081 and moves the cancel state machine to a design PR. Lane B remains on seq2 until a landed, armed head exists. The PR is marked ready with the push; the separate mover child stays held until landing.

Additional Follow-up: REST-terminal background cancellations reach the controller sink before trade_updates reaches the native client. Trigger: the owning cancel-state design work or a measured delayed-stream terminal observation. Source captures, lifecycle notes and the CPython citation correction remain the existing Follow-ups. Hostname hygiene is no longer a finding under the owner's 22:25Z ruling. No actual credentials, broker operation, arming or paper acceptance run.

Final sealed validation at09a14269: native1600/1600 PASS/no skips489.360s; CI-like1600 OK/455 native-only skips459.914s. Structural14/14 native PASS53.689s; focused226 native PASS88.771s/no skips; focused CI226 OK/101 native-only skips78.855s. Three repeated SELL/recovery pairs6/6 PASS30.169s. Ten-receipt/source-map12 PASS0.551s. Four hook tests plus complete blind-checkout45 PASS/no skips124.884s. All96 sanitized log excerpts resolve to public preimages. Six private-prefix inputs resolve; zero tree/body matches. No historical receipt input hashes were rebound.

scripts/validate.py on this unre-based head is rc1, solely for inherited claude-fable-5-1 release freshness; no publication/hash integrity errors remain. #1101 fixes that row on main, and the CC explicitly reserves rebase for the landing cue. This is not a current validator PASS or landing-ready claim. At that cue, preserve every non-registry path patch-id from this read head, regenerate both registries with all current-main rows retained and byte hashes verified, and record merge-tree rc0 before pushing the rebase. The strict source-hash lander then requires the CC's one micro read wave on the rebased head, as ruled02:35Z.

  • CC micro at 9d43e30, P3: completion log head labels. The four published completion logs (review-landing-rebase/completion-*.sanitized.txt) carry "head": "0c65ee7fd…" but ran on that tree plus the staged commit-11 content (disclosed in overnight-landing-rebase-validation.json at :173, :199 and :244). Label them "0c65ee7fd plus staged completion" in the next record; do not cite them as a pass at 0c65ee7.

@seathatflowsinourveins seathatflowsinourveins added the lane:trading Trading lane: sim, paper and live north star label Oct 13, 2026
@seathatflowsinourveins
seathatflowsinourveins force-pushed the foundation/overnight-arming-20261013 branch from ea0a51a to 8f64d5a Compare October 13, 2026 21:08
@seathatflowsinourveins
seathatflowsinourveins marked this pull request as ready for review October 14, 2026 03:04
@seathatflowsinourveins
seathatflowsinourveins force-pushed the foundation/overnight-arming-20261013 branch from 09a1426 to 02e19f1 Compare October 14, 2026 04:32
Use the supported BOATS REST path with fresh carrier, quote and lifecycle guards.
Transient entry dispatch denials leave the node and other held positions alive;
exit-residual refusal retains the node-stop handoff.

Under the CC's 06:11Z P1-B ruling, held positions retain reduce-only PRE/RTH/POST
exits after the 04:00 ET overnight boundary. PRE/POST use extended-hours LIMIT
DAY; RTH uses LIMIT DAY without extended hours. Existing held-quantity guards
also permit bounded explicit recovery. New entries remain refused outside
OVERNIGHT. The opted-in REST quote path selects SIP for these residual exits.

Keep the strict carrier freshness bound, refuse final transport readiness before
journaling wire submission, and retain synthetic evidence separately from broker
acceptance. Paper acceptance remains NOT_RUN.
…entory

Derive the one missing exact repository grant through scripts/local_pages_policy_grants.py at f8d11a6. Preserve every existing grant and publish the inventory regression evidence: two assertion failures before the generated row, all 60 policy tests passing after it. Trading runtime and test code are unchanged.
Consume the actual Alpaca cancellation-rejection event, keep venue halts scoped to OVERNIGHT, adopt the companion scan trade-date/rule primitives, and dispatch bounded cancellation tasks outside owner ticks with joined shutdown. Preserve published history under CC correction 216. Exact 448 base: seven tests fail fifteen assertions with zero errors; native related 726 and CI-like 726 pass after correction, with eight mutation plans killed. Publish sanitized run preimages and exact historical public source inputs, plus a generic seven-character prefix regression without a blacklist.
Propagate failed cancellation outcomes as bounded TransportError so the owner releases its pending latch and retries without a rejection stream event. Keep confirmed terminal 404/422 responses and successful DELETE acknowledgements as data. The production transport/controller regression fails at ea0a51a and passes after correction; targeted native transport132 and CI-visible cancellation/publication/source13 checks pass. Preserve exact public execution-input preimages needed by the approved main integration.
@seathatflowsinourveins
seathatflowsinourveins force-pushed the foundation/overnight-arming-20261013 branch from 02e19f1 to 9d43e30 Compare October 14, 2026 05:32
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

Parked at 9d43e30 (2026-10-14 20:17Z). The co-op is recording the command center's ruling. The command center parked this PR under its bounded-rounds rule, after the review of fix round 8 returned a P1.

The P1, by locator only: a publication-rule-19 occurrence at tools/local-pages/source_policy.json:1590, and its historical preimage copy under evidence/artifacts/overnight-arming-20261013/review-landing-rebase/historical/.

What happens now:

  • This PR gets no further fix round, and it stays a draft.
  • It can re-open only as a new, re-scoped PR, on the command center's cue.

@seathatflowsinourveins
seathatflowsinourveins marked this pull request as draft October 14, 2026 20:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:trading Trading lane: sim, paper and live north star

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant