run_json: verify and copy path-sink equity records in bulk - #553

…f and on

test_selected_validation_reads_only_the_first_equity_record counted the
sink.records("E") passes and expected the anchor check plus the writer's full
pass; test_strict_reader_and_trade_pass_share_one_parser_each expected every
record, equity included, to go through RecordParser.parse. Both describe the
strict reader. With the bulk equity path the writer copies the equity array
from the raw sink bytes and canonical equity lines are verified in bulk, so
neither call happens for them.

Both tests now run twice. The strict row keeps the original expectations
unchanged. The bulk row expects only the anchor pass of records("E"), and in
the parser test every non-equity record parsed once, in file order, through
the one shared parser.

Not run on the author's machine.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…the strict reader

The bulk equity path accepts only canonical lines and replays everything else
through the strict reader. This file attacks that claim with generated inputs:
40 deterministic seeds (fixed integer seeds, no clock, no urandom) of valid
sinks with interleaved trade, event, output and constant rows, and 39
mutation kinds applied to the sink bytes (single kinds and random chains of
one to three), each with the digest and receipt recomputed ("fresh") or left
stale. For every case the report entry runs once with the bulk path off and
once on, on the same file and window, and the two must end alike: the same
bytes and write-call boundaries, or the same exception class, text, typed
code and arguments. The strict reader must also fire the check the mutation
is named for, so the rows cannot pass by corrupting nothing, and a count of
json.loads and per-point decodes proves the bulk path really engages on the
valid sinks.

Three controls show the comparison can fail: a copy of the bulk acceptance
without the repr-identity proof, a grammar that also accepts an integer -0,
and a bulk path that never engages must each be flagged.

Not run on the author's machine.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…-range proof and the reader bookkeeping

The differential fuzz of the bulk equity path gains the cases the review of the first fuzz commit left open:

- faults placed around the 64 KiB page boundary of the sink file (on the boundary line, before it, after it, and at rotating positions), each at the production window and at a 97-byte window, so a line carried across the page edge is compared with the strict reader;
- control D: a bulk proof without the +-(2**53-1) time range test is detected;
- control E: a bulk reader that skips its own record count or line number after an accepted batch is detected.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

The bulk equity path matched a whole 64 KiB window of canonical E lines with one
regular-expression call. The engine keeps the backtracking state of a repeated
group on a heap stack until the call returns, about 2 KiB per line here (every
alternation and bounded repeat of a line nests a frame). A full window of
32-byte lines is 2,048 lines, so that stack alone reached about 4.4 MB, while a
1,000-point sink, one run of 1,000 lines, reached about 2.2 MB. The traced
Python peak therefore followed the window and the file size, and the selected
sink memory row saw a fixed 2.3 MB step between its 1,000-point run and the
larger ones.

The run pattern is now (?:LINE){1,128}: a window is consumed in batches of at
most 128 lines, so the stack and every list built from a batch stay a few
hundred KB whatever the window or the file. The proof, the accept-only rule,
the exact fallback to the strict reader, the 64 KiB geometry and the write
boundaries are unchanged; a failed proof now replays at most one batch instead
of a window.

Tests: a tracemalloc test over 1,000 and 20,000 point sinks through the verify
and copy passes (each peak below 1 MiB, the two within 256 KiB, every line
accepted in bulk), a mutation control that restores a whole-window batch and
must fail both bounds, and a test of the batch contract. The fuzz control for
the integer -0 grammar follows the new run pattern.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…the replayed prefix

The bulk equity path verifies canonical E lines in batches of at most 128 lines and replays a batch whose proof fails through the strict reader. The fuzz now puts the two proof faults, a negative-zero time and a space at run-start +127, +128, +129 and +256 (positions read off the sink bytes, not hard-coded), at both windows, and a premise test fails loudly if the fixture's runs stop fitting those labels. A fifth label (+256 in the 280-line run) puts the +256 offset on a real batch edge, because the longest run is cut by the 64 KiB page.

The decoded-prefix assertion is no longer skipped for proof faults at the production window: a rejected batch is at most one batch long, so the bulk side must still have skipped the decode of every earlier batch (saved >= (accepted - batch) // 2, with the batch size read from run_json).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>