Backpressure paper streams and distinguish frozen outcomes - #1084

Scope

A bounded quote queue could overflow while the native owner loop was briefly
blocked, then a flat run could be reported as having no signals. Use alpaca-py's
awaited handler contract to backpressure the existing queue and distinguish
observed stream failures as stream_frozen. Keep quote-stale-only quiet-market
ends distinct, preserve pending received order events during reconciliation,
and refuse irreversible observation loss through recovery's final send boundary.

• Rebased base commit: 59899aaf32535032fadc4e208d36e7811195e691 (Separate extended-hours paper startup readiness from order freshness #1097 landed first).
• Lane: lane:trading.
• Owned paths: adaptive transport/runner/recovery, metrics documentation, focused
  regression modules, decision and source/evidence registrations.
• Landing head: 9084d29bc193cfd152d570a15e97ac56b28a848d. Cued rebase retains
  the read implementation at12719fff with exactly one approved transport union:
  Separate extended-hours paper startup readiness from order freshness #1097's selected-benchmark max_age plus this PR's stale-drop counter. The new
  combined test exercises registered start30/benchmark5 with handler8/ping10,
  strict order3 and irreversible-loss refusal for buys and owned sells.
• Per-file default patch ids match for nine original non-registry files. The
  transport default id differs in new context; its zero-context id matches,
  all99 added/deleted lines are byte-equal and full source equals the measured
  union. Both registries retain main/read-head keys and bind actual head bytes.

SOTA sources

• alpacahq/alpaca-py 0.44.0 at cc4cb3b7ba50ae250e621983c2779047fb16bb28, data/live/websocket.py:231-274 and :367-368: SDK awaits dispatch/handlers. trading/stream.py:170-182 does likewise. The existing clients and stream threads are reused.
• Same data source :121-128: supplied websocket params replace SDK defaults. This transport supplies its own bounded max_queue and ping_timeout10.
• CPython 3.12.3 f6650f9ad73359051f3e558c2431a109bc016664, Lib/queue.py and Lib/asyncio/tasks.py: existing thread-safe bounded Queue and yielding producer sleep; no queue implementation is replaced.
• python-websockets/websockets 17.2, legacy/protocol.py, installed source :949-955, :1108-1133, :1215-1253: full buffers pause reads and ping/pong progress is bounded by ping_timeout. Handler waits now cap at min(start_timeout,8s), below caller timeout10.

Evidence-class table

Local commands run

Final landing checks (bounded idle scope, TMPDIR=/var/tmp):

<runtime-python> -B -m unittest tests.test_adaptive_paper_readiness_backpressure tests.test_adaptive_paper_start_readiness tests.test_adaptive_paper_stream_backpressure tests.test_adaptive_paper_recovery tests.test_adaptive_paper_exit_quote_age
97 tests OK,rc0.
<SDK-free-python> -B -m unittest tests.test_adaptive_paper_readiness_backpressure tests.test_adaptive_paper_start_readiness tests.test_adaptive_paper_stream_backpressure tests.test_adaptive_paper_source_hashes_manifest
50 tests OK,rc0/skipped2 optional SDK/native readiness integrations.
python3 -B scripts/validate.py
rc0/11955 hashes.84 source rows match actual bytes; all main/read-head keys retained.
git merge-tree --write-tree origin/main HEAD
rc0 against59899aaf;12 owned evidence rows match bytes, other main rows/lists unchanged.

Earlier read-head evidence retained below; it is not substituted for final checks.

systemd-run --user --scope -p CPUWeight=idle <runtime-python> -B -m unittest tests.test_adaptive_paper_stream_backpressure tests.test_adaptive_paper_recovery tests.test_adaptive_paper_transport tests.test_adaptive_paper_exit_quote_age tests.test_adaptive_paper_mover_native tests.test_adaptive_paper_runner tests.test_adaptive_paper_metrics
472 tests OK, rc0 before registry-only rebase.

systemd-run --user --scope -p CPUWeight=idle <runtime-python> -B -m unittest tests.test_adaptive_paper_stream_backpressure tests.test_adaptive_paper_recovery tests.test_adaptive_paper_transport tests.test_adaptive_paper_exit_quote_age tests.test_adaptive_paper_mover_native.MoverNativeEndToEnd.test_quiet_extended_session_stale_only_is_not_stream_frozen
Independent-reader reference selection:210 tests OK, rc0.

systemd-run --user --scope -p CPUWeight=idle python3 -B -m unittest tests.test_adaptive_paper_stream_backpressure tests.test_adaptive_paper_recovery
SDK-free system interpreter:66 tests OK, rc0.

systemd-run --user --scope -p CPUWeight=idle <runtime-python> -B -m unittest tests.test_adaptive_paper_stream_backpressure
Final classifier module:23 tests OK, rc0.

systemd-run --user --scope -p CPUWeight=idle <runtime-python> -B -m unittest tests.test_adaptive_paper_stream_backpressure tests.test_adaptive_paper_recovery tests.test_adaptive_paper_configurable_ceilings tests.test_adaptive_paper_exit_quote_age
Rebased carry:87 tests OK, rc0.
python3 -B scripts/validate.py
Rebased rc0:11922 hashed files; integrity/scope passed.
git merge-tree --write-tree origin/main HEAD
rc0 against329985de; merged source hashes verified.
gitleaks git . --config .gitleaks.toml --redact --no-banner --log-opts origin/main..HEAD
rc0:4 commits scanned, no leaks.

The consolidated fail-before selection at e6771e01 ran20 tests, rc1 with11
failures including subcases and two absent-counter errors. It reproduces the
quiet extended-hours mover classification,15-second handler wait, post-hook
recovery sell and native wire-guard gap. The unexplained frozen-flag case also
fails before correction and then retains generic needs_attention.

An initial454-test batch had three failures: the terminal receipt fixture needed
the new counter fields; two fill-dependent mover cases failed only in that
batch and pass isolated on both unchanged e6771e01 and fixed bytes. That failed
batch remains retained and is not reported as green. The final472 run above
passes after fixture/reference updates. No broker, credential, GO, unit or
account-state call; all test scratch is isolated.

Decision record

docs/decisions/2026-10-13-paper-stream-backpressure.md

The pending-order counter overlaps the callback-held/queued handoff under the
state lock. Recovery checks observation integrity before and after awaited
authorization; the native final guard checks every registered integrity-loss
reason for buys and sells. Definitive-not-sent handling preserves
recovery_observation_integrity_lost. Fresh owned recovery exits remain permitted
through soft stale/reconnect admission pauses; no order is priced from stale data.

Stale-only quotes cannot prove a broken stream. A recorded quote SDK disconnect
followed by successful re-ack provides the selected separating signal. Other
nonstream refusals retain needs_attention. Stream failures remain sticky through
cleanup. Terminal receipts count blocked events once, stale quote skips and
successful reconnects; orderly shutdown may discard a callback-held event and
that shutdown discard is not separately counted or claimed lossless.

Host evidence

No sustained feed throughput, real reconnect timing, broker acceptance or
deployment is claimed. Private operating survey/receipt locators stay in
coordination. This PR must not deploy to extended-hours stages before the
consolidated correction passes its normal reads and lands.

Checklist

• Existing upstream SDK handler/queue mechanisms reused, no dependency/workflow.
• No credentials, account identifiers, private paths or private packet digests.
• Protected operating roots, sealed records and peer-owned worktrees preserved.
• Registry rebase uses the owning register_file tool and merged-byte hashes;
  original PR rows re-registered, patch ids recorded in coordination.

Follow-ups

• Readiness Separate extended-hours paper startup readiness from order freshness #1097 landed first; this head carries its combined-test obligation. Its budget follow-up#1105 has first review priority; no operating pin is installed here.
• Frozen-stream status precedence over a legitimate overnight hold remains separately scoped.
• Reconciliation acknowledgement exceptions can still surface generic needs_attention.
• Shutdown callback discards need a separate counter if operational analysis requires one.
• Sustained native stream throughput/reconnect measurement remains unperformed; do not treat synthetic dispatch tests as broker acceptance.