run_json: buffer the jailed and fd report text as UTF-8 bytes - #559

…62 pinned names)

Both the report-memory change and the paged array store added one pinned slow
row each on top of 60, and git merged the two identical edits of the count
into 61. The pinned set now holds 62 names: 43 from the PINEFORGE_PR_SLOW_TESTS
loop and 19 direct CMake LABELS assignments.

The test assertion on len(PINNED_SLOW_TEST_ROWS) and the four sentences in
docs/ci.md that state the count and its split (introduced by 3ea5a0d, ci:
split the long test rows, shard the full CI legs and run them on the larger
hosted runners, #544, and rewritten by the report-memory change) are updated
to 62 and 43 + 19. No other text is removed.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

… and 19

test_documented_slow_row_counts_match_literal_population pins the three
numbers that docs/ci.md states for the pinned slow rows (all, from the
PINEFORGE_PR_SLOW_TESTS loop, direct LABELS assignments) and two sentences that
repeat them. The previous commit moved the documented count from 61 to 62
(43 + 19) but left this test at 61, 43 and 18, so test_ci_preflight_slice_6
failed on every profile. The test now states 62, 43 and 19. Only those literals
change.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

… copy

The bulk path-sink equity copy added code above the report-sink helpers in
docker/run_json.py, so the three line ranges the memory-management page cites
for _ReportSink, _build_sink_report and _prepare_stream_report no longer
contained their symbols (doc-anchors: SYMMISS). The ranges now read 6183-6220,
7102-7148 and 6983-7003. Citation numbers only: no sentence is added, removed
or reworded.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…StringIO

The jailed sink-off route and the fd route built the report text in an
io.StringIO and then read it back for publication. CPython's StringIO keeps the
written text (one byte per ASCII character) and realizes a four-byte-per-
character buffer at its first read, so the resident text was about five times
the stdout bytes. The text is now built in
io.TextIOWrapper(io.BytesIO(), encoding="utf-8", newline="\n"): the same
characters and the same bytes on stdout, one byte of resident text per output
byte for the ASCII json.dumps output, and non-ASCII text still encodes as
UTF-8. Serialization still completes before the buffer is returned, so a
serialization error publishes no report prefix; _write_report is unchanged. The
remaining io.StringIO in run_json.py serves the capped failure diagnostics and
is not touched. The memory-management page states that the sink-off jailed
route is O(output) and that --report-sink-fd avoids the full report text buffer.

Measured on build hosts (kernel ru_maxrss, stdout byte-equal in every pair):
the zero-trade codegen module (63 B/bar of stdout) has a slope of 363.0 B/bar
for the previous code (five runs on two hosts) and 191.0 B/bar after this
change; the trading module (143.88 B/bar of stdout) 767.4 and 289.7. The
previous slope is exactly 48 + 5 x stdout bytes per bar (the chart plus one
byte per character plus the four-byte-per-character buffer). What remains after
this change is not decomposed here.

Tests: docker/run_json_stringio_test.py. The allocation rows trace prepare plus
the real _write_report read-back on two documents (one 8 MiB string, and a
header with 200,000 small equity records), both profiles: ceiling 2.5 times the
encoded document, derived from the two mechanisms (the previous code is at least
5 times by construction, measured 5.000; the new code about 1.5 times). A slope
row compares an 8 MiB and a 16 MiB document: at most 2.5 per added byte (previous
code 5.0, new 1.05). A diagnostic row prints the peak of one 1 MiB read. Further
rows: stdout bytes equal to the previous code for zero-trade, trading and
non-ASCII input, a mid-document serialization error publishes nothing, mid-write
OSError and short write keep the accepted prefix with a stderr-only reason and
status 1, and the fd route keeps the bounded summary. The rows were red on the
previous preparer and green on this one on a Linux box with Python 3.12.3 and
inside the release image (Python 3.11.2, image pinned by digest
sha256:8a81b6f43944634081044fc7dfff0d18de9ee3d60b476ca785406609dbdb3484).
docs/design/run-json-report-text-rows.md records the commands, the measured numbers and the
history of the allocation rows (the first version did not fail on the previous
code; the ceilings tried after the first box runs were replaced by the
structural one).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Adds a section to the memory-management page for a host that starts the release
container (or docker/run_json.py) and reads the report from an inherited
descriptor instead of a file: which image forwards PINEFORGE_REPORT_SINK_FD (the
entrypoint reads it from 1.8.0 on; a 1.7.0 image ignores it), what descriptor
numbers a caller may pass, the stream's framing and its verified trailer as the
completion marker, what stdout carries, and what a closed reader produces, as
measured on a FIFO and on a stream socket. A statement the tree does not settle
says so.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

The bulk path-sink equity copy added code above _prepare_stream_report and
_prepare_report_json, so the two ranges the memory-management page cites for
them (the scratch_io raise and the _prepare_report_json definition) moved. The
ranges now read 6995-6998 and 7006. Citation numbers only: no sentence is
added, removed or reworded.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…the page now says everywhere

The report-memory text of the memory-management page added by the report-memory
change (memory: state the report-memory bound exactly and add the proofs for it
(resident, disk, jailed arena)) still said that the jailed sink-off route holds
the report text at about five bytes per stdout byte (one byte of text plus a
four-byte-per-character buffer, 363 B and 767 B per bar), in the sink-off
paragraph and in the report-memory table row. That is the behaviour before
the UTF-8 byte buffer. Both places now say: about one byte of resident text per
output byte plus bounded encoding and read-back buffers, measured at 191.0 and
289.7 B per bar (down from 363.0 and 767.4), the remainder not decomposed. The
design document keeps the base figures as the measured baseline.

Removed text, named: the "about five bytes per stdout byte" sentence and the
363 B / 767 B per bar example in the sink-off paragraph, and the "about five
bytes per stdout byte" clause of the sink-off table row, both introduced by the
report-memory change named above.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>