report sink: read trades and equity in one raw sequential pass each - #529

…usal; trailer straddling a window

Adds guards for the new single-scan trade pass: an EIO or a mid-scan
truncation while the scan reads must come out as the typed "report sink is
invalid" refusal with the descriptor released (never a raw OSError, never
"changed after verification"), and a trailer that straddles a 64 KiB window
boundary must stop the hash exactly at the trailer start; an unterminated
trailer there is a format refusal.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…hashes and parses the same bytes

records("T") no longer re-parses the whole verified report sink with the
strict reader. It opens a fresh raw uncached file, reads it in the existing
fixed 64 KiB windows, feeds every byte before the trailer into one sha256,
and cuts the T lines (plus the header, parsed once) out of that same stream
with bytes.find and a bounded carry; only those lines go through a new public
pf_sink.RecordParser, built once per header and used by SinkReader as well.
The scan compares its digest with the one pass A verified, then its T count
with pass A's count, and refuses with typed texts distinct from the strict
reader's (digest differs / T scan disagrees with its verified count / file
identity differs); an OSError during the scan is the typed "report sink is
invalid" refusal. Pass A, the other record filters and the output are
unchanged. Memory stays flat in bars and trades.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…edge for the trade scan

The trade pass finds T lines with a bounded carry across fixed windows. Pin it
at every offset from WINDOW-14 to WINDOW+14, for the first T line and for the
trailer start (which then straddles into the next window): each sink holds a
padding row, then literal and escaped-tag E and T rows, and the trade pass
must equal the strict reader's T rows with A's count and no refusal.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…ss open

records("T") now opens a fresh raw uncached file itself (raw=True) instead of
a strict SinkReader. The test that wraps _open_uncached_sink forwards the
keyword, counts strict and raw opens separately (8 strict: verify, six
filters, the tail E read; 1 raw: the T filter; total equals their sum), keeps
the file-order equality for the raw route, and asserts the raw open is a fresh
unbuffered FileIO at byte zero that is not the strict reader's file.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

The stream-fastpath tests stub run_json._open_uncached_sink with a strict-reader
factory. The trade pass now opens a raw uncached file itself, so the stubs
serve it with real bytes: a raw file over the serialized sink (fresh descriptor,
position zero, bounded reads, counted closes) while the strict route and the
frozen old reference (_OldSink, unchanged) keep the strict reader. Strict and
raw opens are counted and asserted separately, with the same byte-equality
assertions as before; the fabricated receipt digest becomes the real digest of
the serialized prefix so production's digest comparison sees consistent data.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…de-scan sweep

The sweep placed the literal and trailer needles at every offset around the
window edge but the 4-byte escaped-tag needle only hundreds of bytes away. Add
the escaped edge: the first T line of the sink is the escaped one, at every
offset from WINDOW-14 to WINDOW+14, with its byte position asserted.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

… of the window sweep

The escaped edge swapped two tail rows, which reordered the trades and made
pass A itself refuse the sink (trade rows out of report order). Keep the row
order and escape the first T (which sits at the edge) and one E instead, with
the second T and the other E kept literal.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

PR Release runs ctest -L '^fast$' without pytest (ci_preflight.py), which
failed engine #517's pytest row with that label; #522 removes it there.
Same one-line fix for this row, before the F1 PR exists.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…ages

Comments, docstrings and three assertion messages only: they name what the
code does instead of the work item it came from. No executable logic changes.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…equential scan (red first)

Specifies, before the change, how run_json's equity pass reads the verified
report sink: the trade scan generalised to E, one fresh raw sequential file
whose raw chunks go into a single sha256 and from which only E lines are cut and
decoded by a verified-bytes decoder (valid only for bytes whose digest equals
the digest pass A computed in a full strict read). Rows pin the parse count
(one header parse, independent of the E count), the typed refusals (digest
differs, E scan disagrees with its verified count, file identity differs)
distinct from the strict reader's, a differential corpus against the strict
parser (types, -0.0, float spelling, I53 bounds), hostile bytes mapped to typed
refusals, pass A not weakened, window-edge sweeps, provisional-row discard and
early close.

The one F1 row that pinned the old strict error for E is narrowed to None and K;
E moves to the new rows.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Tests only. R1: the zero-byte-body pipeline half expects the digest text,
not a blank-line refusal that the scan cannot reach. P2: the non-list and
wrong-tag shapes move to direct decoder rows. C3: every accepted float
token comes from repr(), with named non-repr alternates per column.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…rict walk

Pass C (records("E")) now shares F1's raw sequential scan: the tag, needle,
decoder and count key are parameters of _tagged_records, T keeps its
behaviour. RecordParser.parse_verified decodes an E line after pass A has
verified the sink: same framing texts, same _field and token hooks, no
lexical byte walk. Refusal order is framing/EOF, digest, count; pass A is
unchanged. The wrong-tag row pins the decoder's own text, and a
whitespace nit in the trade-scan sweep is fixed.

The stream-fastpath rows that pin the old strict pass-C accounting follow
in the next commit.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

PR Release runs ctest -L '^fast$' without pytest; engine #522 removed the
label from the #517 pytest row, and the trade-scan row follows the same
rule. Same fix for this row.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

The report writer now opens the sink once strictly (pass A) and twice raw
(pass B for trades, pass C for equity). Re-derive the stream-fastpath
budgets from that design: strict and raw opens are counted per tag by the
single open_sink factory, successful unnamed reports must open one raw
stream per writer, and the equity-writer trailer row expects the digest
refusal text. Every equality stays an equality, the frozen _OldSink and the
report byte assertions are untouched, and the collected rows are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…ity tests by behaviour

records("E") now opens one raw stream, so the report-sink method row expects
three raw opens (trades, equity, the tail equity read) and six strict opens,
and the changed-after-verification row expects the digest refusal text for
the equity filter. The equity-scan file and its cmake fragment describe what
they test instead of the work item they came from.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…y-scan assertion message

Message text only.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…writes

The trade pass now refuses a sink whose inode or size differs from the one
recorded at verification. This test deliberately rewrites the sink and
replaces its digest and counts to simulate a newly sealed short equity
sequence; it now replaces the recorded identity the same way, so the trade
read-back it asserts still sees the sealed file.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>