Skip to content

Preserve retained paper budgets across startup and recovery - #1105

Draft
seathatflowsinourveins wants to merge 4 commits into
mainfrom
foundation/paper-readiness-budget-handoff
Draft

seathatflowsinourveins wants to merge 4 commits into
mainfrom
foundation/paper-readiness-budget-handoff

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

Keep the retained root request window authoritative before startup HTTP and
through trading/recovery, including a startup observer created before another
invocation first adopts the root. An exhausted200-request root admits zero
GETs from that observer. Recovery also imports sibling request history without
reading or comparing another account's frozen request-cap identity.

This last correction synchronizes every startup callback, first adoption and
replay through the existing native flock pattern. Genuinely unbound default
broker attempts retain a conservative60-second adoption hold in a coordination
marker, so in-flight or not-yet-replayed sends cannot disappear at first binding.
No request/trading journal or trial is bootstrapped for that default path; its
data-GET policy remains unchanged. A contending startup or premature adoption
refuses before HTTP, with no automatic retry.

Sibling journals contribute only retained request/import rows and source binding,
under the existing destination-to-source transaction order. Their risk envelope
is never selected, adopted, compared or changed. The root envelope and current
invocation's exact cap govern startup separately. The two-account regression
retains distinct valid caps and a partially filled owned order, then reaches
startup, replay, cancellation and flatten with both frozen limits unchanged.

Earlier corrections remain:180 failed+12 current startup reservations leave
eight slots; import/reopen/retry does not duplicate rows; linked lane-B recovery
follows cap-only source changes without changing frozen trading/risk limits;
handoff snapshot clocks are captured after the source lock; exporter/dashboard
observe the enforcing source instead of a stale audit mirror.

SOTA sources

Evidence-class table

Claim Evidence class Evidence
An in-flight unbound observer rebinds/refuses after root adoption synthetic/local_integration Real GuardedSession/preflight interleaving: six GETs before fix, zero after199 imported+one reservation exhausts200
Another account's caps cannot gate owned recovery synthetic/local_integration Two-account partial-entry fixture fails persisted_risk_limits_differ before fix; after fix reaches startup/replay/cancel/flatten; SQL tracing finds no sibling limits SELECT
Pending unbound attempts survive the preflight/replay gap synthetic Adoption refuses through59.999seconds and admits at60; default data GET does not extend the broker-attempt hold
Pre-send selection/adoption/replay share synchronization local_integration Actual native flock contention and replay contention refuse before journal creation/HTTP
Retained history/import/cap rotation/telemetry corrections stay covered synthetic/local_integration Existing fail-first and positive regression suite, real SQLite journals and fixture ports
Native broker/session acceptance or provider-wide limits not claimed No broker, credential, GO, STOP or operating-account call

Local commands run

Module runs use pinned runtime Python with -B, idle CPU scope and
TMPDIR=/var/tmp. All identities, HTTP, order ports and databases are fixtures;
telemetry uses loopback only. Exact argv, actual exits, hashes and retained
failed controls are recorded in the sealed correction receipt.

runtime-python -B -m unittest tests.test_adaptive_paper_readiness_handoff tests.test_adaptive_paper_start_readiness tests.test_adaptive_paper_lane_a_continuity tests.test_adaptive_paper_exploratory_admission tests.test_adaptive_paper_safety tests.test_adaptive_paper_runner tests.test_adaptive_paper_credential_source tests.test_adaptive_paper_runtime_lock tests.test_adaptive_paper_recovery tests.test_adaptive_paper_transport tests.test_adaptive_paper_native tests.test_adaptive_paper_metrics tests.test_adaptive_paper_live_manifest
691 tests OK/no skips,rc0, before the final replay-lock regression was added; production bytes unchanged afterward.
runtime-python -B -m unittest tests.test_adaptive_paper_readiness_handoff tests.test_adaptive_paper_start_readiness tests.test_adaptive_paper_lane_a_continuity
78 tests OK/no skips,rc0; all7 changed input hashes unchanged before/after this final run.
python3 -B -m unittest tests.test_adaptive_paper_readiness_handoff tests.test_adaptive_paper_start_readiness tests.test_adaptive_paper_lane_a_continuity
78 tests OK/skipped2 declared optional integrations,rc0, including final replay-lock regression.
runtime-python -B -m unittest tests.test_adaptive_paper_exits tests.test_adaptive_paper_extended_exits tests.test_adaptive_paper_source_hashes_manifest
133 tests OK,rc0.
python3 -B scripts/validate.py
PASS11954 hashes,rc0. All83 source-map rows and actual changed evidence rows match file bytes.
gitleaks dir --redact --no-banner --config .gitleaks.toml blueprints/us-equities/adaptive-paper
rc0.

Both final P1 tests fail before production edits and again using exact reviewed
937fd8f runner/safety source: one assertion failure with six GETs and one
persisted_risk_limits_differ error before owned recovery. The fixed core55 passes,
then the final replay-lock variant is covered separately on the final test bytes.
The first53-test development run retained two failures reflecting the new
pre-send validation point and coordination-only artifact; the tests now assert
missing-source refusal at the callback and absence of every request/trading
journal and trial on the unbound refusal path.

Earlier failed controls remain retained: six/twelve sequential startup sends
at4cd05154, three local recovery/snapshot/telemetry fail-before cases, the
replay-fixture clock correction, the eager disabled-stub limits correction,
and the unchanged two-second native synthetic runner scheduling failure also
reproduced on exact previous production. No order-contract, promotion gate,
native fixture, clock guard or timeout was relaxed to obtain a pass.

Decision record

docs/decisions/2026-10-14-paper-readiness-budget-handoff.md records the ordering,
native lock/source provenance, conservative adoption refusal, sibling-history
boundary, earlier corrections and evidence/qualification limits.

Host evidence

No host acceptance claim or operating deployment. Previously sealed series,
configs and units keep their old pins until explicit successor records.

Checklist

  • Existing SQLite/native flock/SDK mechanisms reused; no dependency/workflow/billing change.
  • No credentials, real account identifiers or private operating paths published.
  • Actual source/evidence bytes registered with the owning tool.
  • Explicit paths staged; ordinary commit/push hooks retained.
  • Peer worktrees, original engine receipts and sealed records preserved.

Follow-ups

  • Keep the runbook guard until merged landing and an operator successor binding: starts at least30seconds/account, preflight refusal ends the series without automatic restart, no lane-B cap changes in the guarded series.
  • Lane-B mover startup/replay integration is a separate P1 prerequisite before any bundle containing this retained-window implementation runs mover trials. Today's sealed older-byte lanes are unaffected.
  • Pre-lock request timestamp ordering can still conservatively refuse; the corrected handoff snapshot does not prove every reservation interleaving. No excess send is claimed for that clock case.
  • Schema-less sibling/provenance and ancestor-safe opens remain separate hardening work. Participating same-root startups must use this coordination; old concurrent implementations are not qualified by it.
  • Historical source/measurement captures remain missing; current mirrors and synthetic controls do not replace them.
  • Actual broker/session restore, exact-byte recovery, native cancellation/flattening and provider/account-wide measurements remain qualification work.
  • Backpressure paper streams and distinguish frozen outcomes #1084 combined readiness/backpressure and exact supplemental two-destination/commit-fault variants remain separate coverage; the quote-age fixture retains real asyncio scheduling.
  • The unchanged two-second synthetic native runner fixture has a reproduced baseline scheduling/cleanup failure, retained independently of the passing final module outcomes.

@seathatflowsinourveins seathatflowsinourveins added the lane:trading Trading lane: sim, paper and live north star label Oct 14, 2026
@seathatflowsinourveins seathatflowsinourveins changed the title Carry retained readiness requests into trading and reopen expired caps Enforce retained paper request windows before startup HTTP Oct 14, 2026
@seathatflowsinourveins seathatflowsinourveins changed the title Enforce retained paper request windows before startup HTTP Preserve retained paper budgets across startup and recovery Oct 14, 2026
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

Parked at 72d4a41 (2026-10-14 17:05Z). The co-op is recording the command center's ruling. That ruling applied its 15:15Z rule for this PR: one last round, and any P1 after it parks the PR.

Why it is parked. A local review of this head found an order-safety P1 in the last-round delta, and reproduced it.

  • blueprints/us-equities/adaptive-paper/safety.py:1828-1829 captures the reservation timestamp before it acquires the shared write lock.
  • Linked trading and another same-root startup can reserve at the same time. A request can then commit to the root journal after that timestamp was taken.
  • The source then raises request_clock_moved_backward, even though the clock moved forward.
  • before_request and recovery treat that as a hard failure, so an ordinary readiness read can abort a cancellation or a flattening.
  • In a local reproduction through GuardedSession, a root read committed 43 µs after the cancellation timestamp prevented the DELETE.

The same review also found a P2. import_request_window (safety.py:1755-1757) looks up (origin, request_id) in a correlated NOT EXISTS, but no index covers that lookup; the table indexes only (origin, source_id). The cost grows with the retained history while both writer locks are held.

What happens now.

  • This PR gets no further fix round. It stays a draft.
  • The runbook guard stays in force: no lane B cap change between opted trials.
  • The final Astra read of this head is still being recorded with the park.

Re-opening needs a NEW, re-scoped PR, built from a design note. The note must state the lock order: take the reservation time after acquiring the shared write lock, and check the clock against committed rows under that same lock. The new PR needs:

  • a test that fails before the fix, for each of: this P1, the two P1s from the previous review round, and the missing index;
  • a fresh Astra read and a fresh command-center review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:trading Trading lane: sim, paper and live north star

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant