scratch: keep a run's command scratch resident up to a fixed size; sync the report sink once at close - #557

Command scratch: the first 64 KiB of each store's extent now lives in the store's
existing page buffer instead of reaching the scratch file on every write. The
first page past that size copies the whole prefix to the file once and the store
then behaves as before (write-through file with the read cache). The fixed
bound is 3 x (64 KiB + 2 KiB) per run and is added to the arena-bound ledger.
Logical reads, writes, charges, typed failures and the byte budget are unchanged;
tests that inspect the scratch descriptor directly now select the all-file
construction explicitly.

Report sink: the per-window pwrite and ftruncate stay; the per-window data
synchronisation moves to one synchronisation when the trailer is written (and on
abandon), before the digest is copied into the trailer, so a failed synchronise is
still reported by the same call with the same error before any report is
returned.

On the 105,159-bar fixed-id benchmark the engine call drops from about 1.59 s to
1.21 s; pwrite64 calls fall from 200,314 to 11,436 and fdatasync from 250 to 159.
Report and sink bytes are identical.

The source allocation-bound gate indexes the sites the resident prefix added or
moved in the command scratch files: its reviewed site inventory follows (13
identities replaced by 14, nothing else), two new unreviewed entries cover the
reserve and the resident flag, and the unreviewed ceiling rises by those two
(1264 to 1266); the real-tree test pins the new open count.

Two citations that prose words introduced after a comma now carry their symbol
in backticks before the line anchor.

The design page and the memory-management page replace the old read-cache
wording with the resident-prefix ledger. Sentences replaced, named for the
revert check: "has no kMaxResident input or native journal-page term",
"changes neither its per-user storage terms nor its command-store shares",
"The bound codec uses the existing command-store share", "supplies no new
private RSS/AS certification" (introduced by 91353de) and "up to 64 KiB of page
bytes per store on the heap", "Counted in bytes; independent of extent and
bars" (introduced by 6d2bf95).

The sink-sync test's counting allocator routes the nothrow allocation forms too,
because AddressSanitizer intercepts them.

Co-Authored-By: Claude Sonnet 5.5 [email protected]

Test note: test_report_sink_writer_memory and test_run_json_bounded_transport read page residency with mincore; on the Cloud Run overlay filesystem they fail on unmodified main too and pass on a disk-backed VM on main and on this branch.

Supersedes #549 (same change, one commit on current main).

Stated limits

1. "Typed failures unchanged" holds for the failure type and its setup only. A command store of at most 64 KiB now lives in memory and can no longer see scratch I/O errors; a failure that used to appear at the first write now appears at the spill, when the store grows past 64 KiB.
2. Report sink durability: the data synchronisation moved from every 64 KiB window to one synchronisation when the trailer is written (and on abandon). A crash between windows can lose the last unsynchronised windows of an unfinished report; a finished report is synchronised before its digest is returned. This is a deliberate trade for fewer system calls.
3. The macOS branch (fsync instead of fdatasync, F_NOCACHE) is not exercised by the new tests, which are Linux-only.
4. Performance: a single run on the 105,159-bar fixed-id benchmark showed the engine call going from about 1.59 s to about 1.21 s with 200,314 -> 11,436 pwrite64 calls. This is one run, not a replicated measurement; it is stated as an observation, not a claim.

🤖 Generated with Claude Code