Memory caches: one shared read-through cache, a versioned per-kind cache policy, an allocation-failure bypass, and cache diagnostics for hosted callers - #670

…n batches (#679)

One merge train of four reviewed changes, built on main after #675:

- Memory caches (#670): one shared read-through cache with a versioned per-kind cache policy and an allocation-failure bypass; the parent terminal scratch test goes back to main's journal refusal row.
- Equity-curve window: a sequential pass reads whole blocks again (the curve read cost and the jailed projection are pinned), the block reset before a block read is pinned, and two documentation fixes.
- Cohort exposure fold: the overflow path no longer walks the whole roster; the completion threshold is pinned to the retained roster size.
- Admission journal fold and exit scratch batches: a new event folds in one validated, hashed pass; live-handle copies publish in batches of 64 rows when no I/O observer is installed; the two rethrows are admitted in the run-failure throw allowlist.

Main is merged once (its tree equals the head this train was built on, so the merge keeps the train tree). Each member had an independent review before joining.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---

* 912b398
Preserve ordered cohort exposure through a private fold seam

Move the exposure fold of a cohort out of PineExecutionAdapter into a free
function over the origin roster and the live-unit map, appended at the end of
the adapter translation unit. The old selector moves verbatim: same visit
order, same additions, same final clamp, so results are bit-identical.

The member overload keeps its exact source footprint, so later line citations
do not move. A values-only property test compares the seam with an independent
frozen copy of the old selector byte for byte over magnitude mixes, repeated
incarnations, holes, retired prefixes, overflow and special doubles.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* b27f854
Observe cohort exposure work and guard its growth

Report the work of every exposure fold through the existing work macro (calls,
unit-row visits, retained window, live rows) from the member wrapper, and add
rows that pin the growth: on one reused entry id the fold is cubic in the
number of openings, on distinct ids it is quadratic. The rows gate the doubling
ratio and absolute bounds, so they fail on the current selector and pass for
any fold that is linear per call.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 4aa1c33
Bound ordered cohort exposure folding with stack sorting

Replace the quadratic position selector by an exact fold: walk the retained
roster window once when it is small against the live rows, otherwise gather the
(position, units) pairs into a fixed stack buffer, sort them by position and
fold in that order. Both paths add the same doubles in the same order with the
same multiplicity and filter as before, so every result is bit-identical; an
empty roster or an empty live-unit map returns zero early. No heap, no cache,
no new member: the fold uses only the existing roster and map.

An opt-in fixture build compares every call with a frozen copy of the old
selector and aborts on any difference in the bytes.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 81ae80f
Cover the stack-sort path of the exposure fold and drop a pass-by-skip row

Add rows on a roster forced onto the sorted path: NaN, infinities, negative
zero, zero, a negative value and the smallest denormal mixed with positive
rows, a maximum-double overflow that must clamp to zero, and interleavings whose
ascending sum differs from every grouped order, all compared bit for bit with
the frozen selector. Remove the caller-group registration from the product test
list (it only skipped without its observation overlay) and extend the oracle
option to the two quiet-bar libraries that compile the adapter.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 27fa59b
Register the exposure fold tests through a lane file and keep the CMake tail intact

Register the two exposure fold test executables from their own lane file instead
of inserting them into the middle of the main test list, and place the opt-in
oracle option immediately before the lane include block, so no earlier line of
the main CMake file moves and the fast-row registration block stays the last
CMake code.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 2d98081
Record the exposure fold's new container sites in the reviewed site inventory

The exposure fold moved out of the adapter class into a pair of free functions at
the end of the adapter source. The reviewed site inventory still held the three
assign records of the removed member function, so the source-bounds gate reported
two stale identities and five new ones, and three tests that run the gate on the
real tree failed with it.

Replace the three removed records with the five records the gate names: the
position, remaining and next assignments of the fold and the position and next
assignments of its checker, each with the scope text and ordinal the gate printed.
No manifest entry changes state and no entry is added or removed, so the entry
and unreviewed counts stay as they are.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 0190528
Admission journal: fold a new event in one validated, hashed pass over its encoding

The admission journal fold used to scan an event record to validate it and then read the same record a second time to hash it, so every appended event cost two reads of each of its pages (16 source reads for a 3,773-byte record on 512-byte pages). A new fused fold, fold_validated_journal_record, validates and hashes the same forward field stream in a single pass (8 reads for that record) on both the bound and the unbound path. The standalone scanner and folder remain for the other consumers.

The result is identical: digest, summary, sequence, rejection gate and the published frontier are unchanged, and a failure publishes the same status and diagnostic as before. After a long-text skip the old scanner's refill alignment differs from the fused reader's, so on a source read failure the fused fold replays the validation once to recover the old error priority; it never retries the hash and never turns the original read failure into success.

The workspace is a fixed constant: a compile-time ledger keeps the fold's own state, diagnostic transport, summaries and controls within 1,024 bytes, with a 256-byte reused field buffer and no new heap allocation. No state survives the call.

Tests: the fold-cost row now counts work (reads, encoder words and writer-status checks per new event, derived from the schema's field geometry and the page size, with the unchanged window costing zero), a failure-publication row corrupts a stored record and checks that nothing is published, and a new differential unit compares the fused fold with the separate composition for all event kinds, text and array sizes, page sizes, malformed encodings and byte-range I/O faults. The schema checker and its mutation control name the new call.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 783016f
Exit scratch: publish the live-handle copies in batches of 64 rows when no I/O observer is installed

The adapter's two flat live-handle copies (the unconditional snapshots taken when an Applied callback finds a flat position) put each handle into the exit scratch as its own transaction, so N rows cost N publications. They now run in one insertion transaction that publishes every 64 rows (the existing exit-scope batch constant) and once more for a nonempty tail: 0, 1, 64 and 65 rows publish 0, 1, 1 and 2 times instead of 0, 1, 64 and 65. The transaction is finished before the consumer loop starts, so no builder is open at Applied, consumer, hash, callback or teardown boundaries.

The batched form is eligible only when the bound backing carries no I/O observer and the context has no row or lookup observer. Production installs no observer (the one source-side registration clears it for derived identities and restores it); observers exist in measurement and test harnesses, which therefore see exactly the charge sequence they saw before. ScratchBacking and CommandScratchStore gain a const has_io_observer query that reads the already-present observer member, and the observer's comment now states its contract: a passive accounting sink that must not call back into the engine, a store or a consumer. All other producers keep the per-row path.

No state is added: the mode is a template parameter and the store's existing fixed transaction workspace does the buffering.

Tests: a construction-publication row counts work (puts, fresh handles, publications) for 0, 1, 64 and 65 rows with no observer, and an observed row requires N publications and the same charge-by-charge sequence as the per-row reference. A test comment now states that 64 is a batch constant, not a page capacity (an ordered leaf at 512-byte pages holds 58 keys).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* a279666
Memory caches: one shared read-through cache, a versioned per-kind cache policy, an allocation-failure bypass, and cache diagnostics for hosted callers

The engine's resident caches (command scratch pages, id-table pages, the trade ledger's decoded rows, feed projections, TA history windows, tail/sequence/curve windows, and the native journal's pages) now share one read-through cache component and one policy. Every cache is a fixed constant chosen at run begin, independent of bar and trade counts; a read that misses goes to the scratch backing, and an optional cache that cannot allocate is bypassed instead of failing the run.

## Cache policy

- A new additive C export, strategy_set_cache_policy_v1 with pf_cache_policy_v1 / pf_cache_policy_entry_v1, sets a per-kind resident byte allowance. Omitted kinds keep their defaults; NULL restores them; duplicate or unknown kinds, nonzero reserved words and an unrepresentable size are typed setting_rejected errors. A run freezes its settings before its first store begin or append. PF_ABI_VERSION and every existing layout are unchanged.
- An allowance is inclusive: it covers each line's metadata, not only its payload. A capacity below one line bypasses caching for that kind; reads still use the backing.
- The trade ledger keeps its private pages (256 slots per page, lazy allocation, growth that preserves entries, release outside the retained window). Its allowance selects the largest row count R with R * 312 + ceil(R / 256) * sizeof(void*) <= capacity_bytes, converted once; the default is 65,536 rows (20,449,280 bytes on LP64) with or without a report sink. A nonzero capacity converting to fewer than one row, or above the 1,048,576-row ceiling, is a typed setting error.
- The native journal now records events of any size across as many pages as they need (the former one-page refusal is gone); the TA history cache, the feed projections, the sequence index and the tail windows account for their line metadata inside their allowances.

## Allocation failure is a bypass, not a failure

An optional cache allocation that fails (the command cache's slab, the ledger's directory or a page) permanently bypasses further cache allocation for that owner and the run continues through the backing, with a typed count. Required allocations keep the typed out_of_memory; a syscall error keeps scratch_io_failed.

## Diagnostics for hosted callers

- A new additive export, pf_report_cache_stats, reads the run's resident cache bytes (each allocation owner counted once) and its allocation-bypass count after report finalization.
- docker/run_json.py adds cache_resident_bytes, cache_bypass_reason ("alloc_failed") and allocation_bypass_count to the report diagnostics only when a bypass occurred; a run without a bypass has the same report bytes as before.

## Tests, gates and documentation

Cache sweep rows for every kind, allocation-failure and fault-injection rows, lazy-cache and allocation-census rows, a layout probe whose pins follow the cache-capacity members, and the source-bounds inventory for the new allocation sites. The two ledger cache sweep tests are pinned in the verifier's slow-label bound. docs/pages/memory-management.md, abi-stability.md and report-schema.md describe the policy, the bypass and the diagnostics; the doc anchors that moved are re-pointed.

Retire the superseded documentation sentences introduced by 819bbac, 197646c, 536f8a7, bd0cc60, 7f14930, df7ddc4, 29a825b, a94c55f, 1654497, 012f70b, cf20c17, faf4977, 6d2bf95, a991b4e, 8713a82, 6963801, 5691220, d093560, 1dd5430, f0b6213, 772766a and b4a2c19 in docs/design/ta-window-cache.md, docs/pages/memory-management.md, docs/design/tail-page-windows.md, docs/feed-projections.md, docs/design/arena-bound.md, docs/design/o1-report-contract.md, docs/design/native-feature-parity.md and docs/adr/0001-kernel-adapter-boundary.md. This explicitly acknowledges their retired no-C-setter and fixed-residency claims, intermediate cache/observer implementations, earlier payload-only metadata accounting and three/five-decorator K formulas, old ledger bounds and constructor-policy wording, the journal's former one-page EFBIG refusal and keep-window description, the retired statement that the verifier test pins the same 118/98, the ledger-cache sentence that writes fill only an existing cache, and the feature-parity design's reference to the ADR section on current state versus target, its note that the independent-host examples option is inert, and its wording of no top-level examples target, and the command-scratch fixed-K formula with its 202,752 B figure and the kCacheBytes page-size sentence (replaced by the shared read-through cache accounting); it also acknowledges replacing "Prefix configuration does not alter rebuilding or query ordering." with "Cache configuration does not alter rebuilding or query ordering." The replacement text documents the shipped per-handle cache setter, shared read-through adapters, per-kind accounting and multi-chunk journal records while retaining backing-read, allocation-bypass and deterministic-result guarantees. In docs/adr/0001-kernel-adapter-boundary.md also replace "nearest-tick rounding, ties away from zero" with the implementation's floor(price / tick + 0.5) rule without an epsilon, and name cache_policy.hpp alongside the other kernel-only includes.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 55cc25a
Merge main into the memory-cache branch

Brings in the configure-time check that a test carrying the fast label must be listed in PINEFORGE_PR_FAST_TESTS. The merge applied without conflicts and changes no file of the cache work.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 878af53
Merge main into the memory-cache branch

Brings in the jailed temporary-file witness test and the strategy runner changes that landed on main. The merge applied without conflicts and changes no file of the cache work.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 2e9c083
Native journal: restore the one-page event refusal; the memory caches no longer carry the multi-page event

A single serialized journal event larger than one spill page is refused with EFBIG and stays resident again, exactly as on main. This removes the extent writer and reader added with the cache work (ExtFile write_pending and publish_append, the ByteWriter extent constructor and finish, the ByteReader extent constructor, the 64-bit remaining count and its representability checks), the continued-page selection and validation in the journal store, the test-only resident-threshold seam and set_journal_residency_for_test, the test that drove oversized events, and its registration. A separate change implements oversized events on its own.

Everything else of the memory caches stays: the tail-window read-through caches and their statistics, the per-kind capacity policy, the successor-root cache, and the ordinary page-index arithmetic hardening.

docs/pages/memory-management.md again states the one-page refusal in the places that described continuation chunks. Retire the sentences introduced by 878af53: "Older events read back through four decoded slots using 32,768 B transfer chunks" (kSpillPageBytes), "A larger serialized event continues across as many chunks as it needs, with one directory entry for the full event", "Page size does not limit event size; scratch backing and address-space exhaustion retain their typed failures", and "An event larger than the 32,768 B transfer buffer is stored across continuation chunks and does not fail merely because it leaves residency". They are replaced by the one-page EFBIG refusal of main.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 89d78da
Tests: feed projections create their backing files in the configured scratch directory, and none for a small default run

Add a Linux source-layer test that watches two directories for the creation of projection backing files (names starting with pineforge-feed-): the scratch directory configured through the report sink, and the default scratch location (TMPDIR). A run with a configured scratch directory and projections one row past their fixed transfer page (the recorded series and the three calendar projections of a daily feed) must create all four backings inside the configured directory and none in the default location. A default run with one-row projections must create no projection backing at all, because a projection that fits one fixed transfer page needs no file.

Every row read back must equal the row supplied: the recorded series bit for bit, and the calendar rows against the values derived from the input stamps. The row counts come from each projection's own page size.

Nothing in src/ or include/ changes; the test is expected to fail on the current tree because the projection builders ignore the configured directory.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 9edaaa7
Feed projections: honour the report-sink scratch directory, and keep a one-page projection out of scratch

The calendar day partition (trade day, week open and month open columns) and the raw-array recorded series built their projections with the default scratch directory, so a configured directory was ignored and every nonempty projection opened a backing file even when it was far below one transfer page. The engine now hands the report-sink scratch directory to those builders.

FeedProjectionBuilder gains an explicit resident mode used only by those production sites. A projection that never passes its one fixed transfer page (4096 / sizeof(Row) rows) is published from an in-memory reader of exactly that page and opens no file. The next append past the page spills through an unlinked ScratchFile in the configured directory; a directory that cannot hold the file raises the existing auxiliary scratch I/O error, with no fallback to another directory. The default builder is unchanged and stays file-backed, so the tests that prove cold reads and cache behaviour through the scratch provider keep their meaning, and a reader supplied by a caller keeps the directory it was built with.

BacktestEngine gains a protected const accessor for the directory, defined inline so no source line that the documentation anchors cite moves, and build_native_day_partition takes the directory as a trailing defaulted argument.

The new test watches the configured and the default scratch locations for projection backing files: four backings in the configured directory and none in the default one for projections one row past their page, none for one-row projections of a default run, and a typed ENOENT with nothing created elsewhere for an unusable configured directory. Row values are read back bit for bit.

docs/feed-projections.md and docs/pages/memory-management.md describe the resident page and the spill directory. Retire the sentences introduced by a279666 in docs/feed-projections.md: "The default producer writes complete rows sequentially to an unlinked ScratchFile, with one mandatory transfer page per producer", "Publication explicitly finishes the final partial page", and "Destruction performs no flush"; the production builders' one-page resident rule replaces them.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 9bd3390
Merge main into the memory-cache branch

Brings in the engine changes that landed on main since the previous merge. The merge applied without conflicts and changes no file of the cache work.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 5b977f3
Source-bounds records and documentation anchors follow the journal cut and the feed projection change

The reviewed site inventory and the bounds manifest return to main's entries for the ByteWriter owning buffer, the journal payload append and the ByteReader cursor, which the restored one-page journal path no longer changes: the ByteReader buffer and cursor assignments and their manifest entry are gone again, and the journal payload append is back in its function scope. The six site identities of build_native_day_partition (tz, stamps, the three projection builders and the late-open assignment) carry the new trailing scratch-directory parameter in their scope.

Five documentation anchors into src/native_order.cpp (the successor-root row assertion and tail bound, index_committed, remaining_path_delivery, check_execution and apply_execution) point at the lines the cut leaves them on.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 86a44b5
Source-bounds records for the feed projection builder's resident mode

The builder now creates its scratch provider either in its constructor (the default, file-backed mode) or in a spill helper once a resident projection passes its one transfer page, publishes a resident provider holding a copy of at most that page, and keeps the scratch directory it was given. The reviewed site inventory records the two provider allocation sites, the resident provider allocation and the directory string, and the bounds manifest gains an entry for each of the last two; the existing provider entry points at its new line. Every entry stays open: nothing here claims a total-row bound.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* fabcfbd
Source-bounds ceiling for the two feed projection builder entries

The previous records commit added two open manifest entries (the resident one-page provider and the builder's scratch directory string) while the restored journal path removed one; the open unreviewed ceiling rises from 1340 to 1341, which is exactly the number of entries now in the manifest. Every entry stays open and no bound is claimed. The ceiling stays below the one on main.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* ed4468c
Make the symbol-feed count and request-feed inventory size limits parameters, default unlimited

The runner refused more than 256 symbols or feeds and the two inventory
readers refused an inventory over 8 MiB, each as a fixed constant. They are
now options: --max-symbol-feeds on the runner, --max-inventory-bytes on the
runner and the inventory binder, forwarded by the entrypoint through
PINEFORGE_MAX_SYMBOL_FEEDS and PINEFORGE_MAX_INVENTORY_BYTES. Unset or 0 is
unlimited, a positive value is the limit, and anything else is a usage error.
With a limit set the typed refusals (too_many_symbols, too_many_feeds, binder
exit 10, the incomplete-inventory admission refusal) are unchanged and name the
actual limit.

The former values live in one opt-in set, selected by --tv-compat or
PINEFORGE_TV_COMPAT; an explicit limit, including 0, wins over it. Without it
nothing is capped. The key-length limit is a name check and is unchanged. The
two new runner options share the "--m" prefix, so the ambiguous-prefix
diagnostic now lists them too; its test pins the new text.

Documentation pages and anchors that cited the moved lines are updated. The
BARS-BIN-INPUT design note drops statements that no longer hold, named here so
the revert check can match them: "All source coordinates below refer to that
tree." ; "At most 256 symbols and 256 total extra feeds across the index." ;
"`_SYMBOL_FEEDS_MAX` docker/run_json.py:N;" ; "The old comment example lists
only CSV feeds; it is an example, not an extension restriction (comment's
`feeds` docker/run_json.py:N)." ; "The actual branch accepts binary feeds
(`suffix` docker/run_json.py:N)." The sentences were introduced by 91353de; the cite `_SYMBOL_FEEDS_MAX` is too
short to match by words, so it is named by that commit here as well. The first is replaced by a statement that
runner and entrypoint coordinates follow the working source; the second and
third by the parameter wording (default no limit); the last two are folded
into one sentence saying the loader accepts CSV and binary feed paths.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* a5dad8a
Source gate test: the pinned open-entry count follows the manifest

The real-tree gate test pins the number of open manifest entries as unreviewed plus pending-flat-memory. The manifest now holds 1341 unreviewed entries (the ceiling raised in the previous commit) and 184 pending-flat-memory entries, so the pin is 1341 + 184 = 1525, which is what the gate counted on the real tree. The pin is derived from the manifest ceiling and is not a typed observation: every other assertion of the test is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 645e8a3
Merge main into the memory-cache branch

Brings in the run-quota admission of scratch-file growth and of the sink assembly's temporary storage. Three files conflicted and are resolved by hand: the source-bounds manifest keeps the cache work's entries and takes main's three new entries under fresh ids (the cache work already uses the ids main took for its own new entries), with the open unreviewed ceiling and the gate test's pinned open count both set to the manifest's 1344 unreviewed plus 184 pending entries; docs/pages/memory-management.md keeps the cache wording and takes main's line numbers for ScratchFile::open and kCacheBytes in the rewritten scratch_file.hpp.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 15dab60
Merge main into the runner limits branch

Takes main's run quota admission change; no conflict.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* ee7b30f
Merge main into the exposure fold branch

Brings in the engine changes that landed on main since the branch was cut. The merge applied without conflicts and changes no file of the exposure fold.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* e13c5ab
Tests: the parent terminal scratch test goes back to main's journal refusal row

The memory-cache change had rewritten the journal limits row of this test to expect that a single event larger than one spill page is accepted across several pages, and added a --journal-limits-only entry for it. That behaviour is not part of this branch any more (the journal keeps main's one-page EFBIG refusal), so the row failed in every full verification profile (test_parent_terminal_scratch, release, debug, kernel and sanitizers) with "native o1 journal event exceeds the spill page byte bound: File too large". The file is restored to main's text, whose row asserts exactly the refusal, that nothing is published by the failed append and that a retry also publishes nothing.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* a334e0f
Name the ambiguous-prefix message change of the limit options and pin it

Registering --max-symbol-feeds, --max-inventory-bytes and --tv-compat adds
them to the candidate lists of the runner parser's ambiguous-prefix refusal:
--t now also names --tv-compat, and --m and --ma also name the two limit
options. The exit status, the failure code and its args are unchanged. The
changelog now states this, and a test pins the exact new text of the --t and
--ma refusals (the --m text is pinned by the existing prefix test).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* f035f95
Cohort exposure fold: the overflow path no longer walks the whole roster

When a cohort has more than 128 mapped positions the fold abandons the fixed stack sort and completes through a walk over every retained roster member. A pinned retained prefix can keep that member count growing with bars while the number of rows and mapped positions stays fixed, so the completion cost grew with history.

The overflow path now counts every mapped position without extending the fixed 128-entry workspace and compares the retained member count M with the selector's own charge (P + 1)(R + P), where P is the mapped positions and R the live rows. Below that charge the roster walk is cheaper and is used; at or above it a private copy of the original selector runs, term for term, so the acceptance-ordered floating-point additions and the returned double stay bit-identical. The reported steps include the gather and the chosen completion. The charge comparison saturates at the diagnostic integer maximum so it cannot wrap; it does not cap any input and allocates nothing.

Two rows are added. The work test builds two windows with P = 129 and R = 1 whose M differs by a full selector charge and asserts the step difference stays within the derived P(P + 1) bound (the old path differs by the whole charge, 16,900 steps). The fold test compares a P = 129 window with M above the charge against the old fold bit for bit.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 59bd663
Merge main into the limits branch

Takes the output series limit parameter that landed on main. The runner
parser now parses through the output-series helper and then applies the
compatibility defaults of the limit options; the changelog keeps both
entries; the memory page keeps main's text with the run_json.py citations
re-pinned to the merged file.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 3da8fbb
Cohort exposure fold: record the selector copy's sites and pin its order and boundary

The overflow completion's private copy of the original selector declares a next and a positions variable and a next inside its position loop; the reviewed site inventory did not know them, so the source-bounds gate named three new assign identities and the tests that run the gate on the real tree failed with it. The three records carry the scope text and ordinal the gate printed. No manifest entry changes state and none is added, so the entry and unreviewed counts stay as they are.

The rows that guard the overflow path were too weak to catch an order or filter error in that copy: one live row of 1.0 units adds exactly in any order, and a path value of 3 is what both completions return. A new fold row uses four live rows with a 1e16 row among ones, derives the only double an acceptance-ordered sum can give (a different order gives another value), cycles NaN, both infinities, negative zero, denormals and the largest double through the finite and positive filter, and pins which completion ran from the step count at M one below, at, and above the selector charge. The work row now also asserts that two windows differing only in M cost exactly the same steps and that the cost stays within (R + P)(P + 2).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 2ee8cf7
Native live startup test: the example exports engine layout stamp 2

The output series limit change raised PF_STREAM_ENGINE_LAYOUT_VERSION from 1
to 2 on purpose (the engine gained a member), and the module-layout assertion
in native_live_startup_e2e.py still pinned version 1, so the native-live legs
failed on main. The pin moves to 2: the normally linked native example must
still export the stamp, and it must still be exactly the version the runner
accepts; an accidental bump stays a test failure.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 6900435
Admission journal fold and exit scratch batches: two reader findings, with their rows

The fused journal fold replayed the old validation scan only when the backing read failed after a skipped text. A schema failure found by the fused pass in the same record was returned as such, although the old two-pass code would have met a read fault on its next page first (its refill boundaries differ after a skipped text), so the status and diagnostic of a record that is both corrupt and unreadable changed. The fold now replays the old scan on any fused failure after a skipped text, still never on a successful record, and keeps the original failure when the replay finds nothing.

In the bounded-batch exit scratch builder, a run-identity failure raised while an earlier append had a batch open unwound through the scope's teardown with the transaction still active, so the teardown began a nested transaction, remembered a programming error and left the transaction open. The builder now rolls back an open batch before raising that failure and on destruction (before the scope erases its rows), finish() releases the batch and rolls it back if its teardown check fails, and a failed put or commit keeps the builder's batch state in step with the store. The class comments no longer claim that every edit commits separately.

New rows: the journal fold across fault offsets after a skipped text compares its status and diagnostic with the old composite built from the old functions; the exit scratch rows cover a run-identity failure at a count that is not a multiple of 64, a put or commit fault in the middle of a batch, and the 63-row boundary, each comparing the exception and the end state with the per-row path.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* e466cc0
Exit scratch batch builder: admit its two rethrows in the run-failure throw allowlist

The batch builder's finish() and fail_batch each roll back or record the state of an open batch and rethrow the exception they caught; the run-failure code check reads a bare rethrow as an uncoded throw unless a reviewed row admits it, so it failed, and the verifier tests that run the real check failed with it. Add the row for the two statements, with the reason: the original exception keeps its own code. No throw is added or changed.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* a1a2abe
Equity curve window: a sequential pass reads whole blocks again; pin the curve read cost and the jailed projection demand

A spilled equity curve went through the shared read-through cache with one 24-byte point per line and no read-ahead, so a forward or reverse pass over N points cost N backing reads where the previous 64 KiB window cost ceil(N / 2730) (a lab run: 5,477 points took 5,478 backing reads against a bound of 5). When a backing miss is adjacent to the previous lookup (the next or the previous point) and the cache is enabled, the window now reads the aligned block of up to 2,730 points containing it with one read into one fixed array of 2,730 points in its state, and serves the points of that block from it. Any other miss reads one point as before; a lone point is kept until a sequential block replaces it, so a distant probe cannot lose it. A failed block read publishes nothing. Zero and sub-point capacities and the allocation-failure bypass read each point directly and never touch the array. The array is part of the window's state, so cache_control_bytes() already charges its 65,520 bytes; nothing scales with points. The default 2,730-line policy, the policy setter, every value and every metric bit are unchanged.

Tests: the curve test counts the backing reads of a forward pass, a reverse pass (also at 2,731 and 5,461 points, where the endpoint probes collide in the default cache) and asserts ceil(N / 2730) plus the two boundary probes, keeps the zero and sub-point capacities at one read per access, bounds the random access byte total by 24 bytes per lookup plus one block for each adjacent pair of the generated sequence, compares every returned point bit for bit, and checks that the extremes fold stays exact. The ordinary heap budget of the selected-metrics test adds the block workspace. The feed projection test gains a jailed case: with an arena attached, a calendar projection and a recorded series longer than one page succeed inside an arena sized from the page and chunk arithmetic, create no file in the configured or default directory, and fail with the arena's typed exhaustion error when the arena is one byte smaller.

Docs: memory-management.md, tail-page-windows.md and feed-projections.md state the block fill, the fixed workspace and what cache_control_bytes() includes, the in-memory limit of one 4096-byte page per calendar column or recorded tape (512 int64 rows or 256 recorded rows of 16 bytes), and that an attached arena supplies the spilled pages; two stale line citations are re-read from the source. Sentences replaced by that wording, named for the revert check: "Each point is a 24-byte shared-cache page." and "The curve therefore adds one metadata charge per point, rather than pretending that all 2,730 records share one tag." (a279666); "Metrics re-read scratch through bounded windows" and "`ScratchCurveWindow` src/report_scratch.cpp:N)." (ff05c78); "2,730 complete 24-byte points; the writer's retry batch is mandatory workspace" (645e8a3); "Production builders keep projections of at most one fixed transfer page in an in-memory reader and spill larger projections" (9edaaa7).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* f8e9589
Merge the memory-cache head into the runner-limits head

The memory-cache head adds the shared read-through cache, the per-kind cache policy, the cache statistics and the policy setter to the C API; the runner-limits head adds the output-series limit functions and moves the engine layout stamp to 2. Both sets of exports and both sets of documentation survive: the runtime inventory is 122 declarations with 102 implementations (the common base's 118 and 98 plus two from each side), pineforge.h declares 111 of them, and the checker, the exporter test, the driver test and the layout probe expect those counts and the combined sizes (the adapter 89,488 bytes, the host 107,944 bytes, derived by hand from the merged headers). The pages that cite docker/run_json.py, docker/entrypoint.sh or the report scratch source take their line numbers from the merged files.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 4c9a6c4
Cohort exposure fold: pin the completion threshold to the retained roster size

The choice between the roster walk and the selector copy compares the retained member count with the selector charge. Every row of the overflow path built a roster whose retired prefix is empty, where that count equals the total number of accepted entries, so using the total in its place went unnoticed. In a run, retiring a prefix makes the two differ: the results stay bit-identical but the work does not, and the wrong count picks the selector when the walk is cheaper.

The new row puts two unmapped entries in front of the mapped block, pads the roster to one more than the selector charge, retires the two entries and so leaves the retained count one below the charge while the total stays above it. It asserts the retained and total counts, the sole large row's absolute position, the order-sensitive total, and that the walk ran (the gather's steps plus the retained count); a threshold on the total would report the selector's steps instead.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* f7ecc1d
Merge the equity-curve window commit into the train

The commit makes a sequential pass over a spilled equity curve read whole 2,730-point blocks again, with the tests and documentation that pin it. One documentation paragraph was edited on both sides: the train's copy carried the shifted line numbers of the writer constants, the commit's copy carried the new description of the reader. The merged paragraph keeps both: the reader description of the commit and the line numbers read from the merged sources (the constants at report_scratch.hpp:20-22, the writer state at report_scratch.cpp:160-168, the reader state at :278-334).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 1d4f883
Merge the cohort exposure fold into the train

* 425ff1d
Merge the admission journal fold and exit scratch batches into the train

Two conflicts. The run-failure throw allowlist gains both sides' rows (the series window's rethrows and the exit scratch batch builder's two), each row unchanged. The native-feature-parity page was edited on both sides only in its line citations; the train's text is kept, and the citations into the adapter source that sit after the two lines this change inserts (one near line 25,856 and one near 26,372) are re-read from the merged source by the documentation anchor check that follows.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* ac04acc
Equity curve window: pin the reset of the block before a block read, and two documentation fixes

The window clears its block before it reads a replacement and publishes the new one only after the read completes, so a read that fails after a short read cannot leave a half-overwritten block marked valid. No row exercised that: the existing failed-refill rows never reach an adjacent block fill that fails. The new row walks the first block through a one-line cache, shortens the backing file inside the next block, expects the typed I/O failure on the adjacent lookup, and then requires an intact early point to read back bit for bit (without the reset it would return the bytes of the partially overwritten block).

The equity writer's citation in the arena bound page pointed at the line that is now another function; it names the writer's write line. The cache parameter page now says what the documented bypass costs: zero capacity turns the cache off, so each lookup reads one point.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 883a842
native-feature-parity: re-read five source citations after the merges

The exit scratch batch change inserts two lines into the adapter source ahead of the cited symbols, and the train's other members moved the reflection helper in the admission source by four lines. The documentation anchor check named five citations whose symbol had moved: the pending-order birth reach (two rows, 27807 to 27809), the fill-quantity probe (27386 to 27388), the risk direction setter (27193 to 27195) and the reflection field helper (658-661 to 662-665). Each now names the line that holds its symbol in the merged source.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* 4f388ba
Merge the equity-curve window follow-up (failed block row, citations) into the train

* f25f92c
Merge main (the runner limits, #675) into the train

Main now holds the runner limits change as one squash commit whose tree is identical to the head this train was built on (the tree of the limits branch's last commit and the tree of the squash are the same object). The train already contains that head and its history, so the merge keeps the train's tree unchanged; a content merge would only re-apply the same lines against the train's later edits to the same documentation hunks and conflict without changing the result.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

---
Merge note (L6 super top): train T4 = #670 memory caches + equity-curve window + cohort exposure fold + admission journal fold / exit scratch batches, built on main aa30961 (T3); head f25f92c, tree d9cd688 (behind main 0, so the squash keeps the gated tree). Merge gate on this head: pineforge/verify 8/8 on tree d9cd688 (job rj-20261031t184042-b7b2a2: release 1487, sanitizers 1354, debug 1382, kernel 718, parity 312/312), pineforge/parity success (sweep exp-engine-pr679-f25f92ce-20261101 paired with codegen main c3ae98eebe44, 7989 probes, cov lost 0, band1/band2 +0/-0, hard reg 0, leavers 0), GKE PR CI 13/13 checks green. Doc-reverts: 0 unnamed sentence deletions vs main (22 pages), and 0 on each of the 28 non-merge commits.
Native-live on this exact head on spot boxes before merge: native shards 1, 2 and 4 passed; shard 3 passed 393/394, the one red being test_native_live_report_memory, which hit its fixed 160 s deadline under parallel box load (T3 on main passed it at 93.77 s on GitHub and 125.15 s / 107.09 s under the same box load; T4 alone on an idle box passed 3/3 at 51.72 / 51.35 / 51.05 s). live-sanitizers 119/119; live-tsan 118/119, the one red being native_live_ratio_interval_e2e, the known timing case already on main (CR-583; solo 3/3).
T3-vs-T4 A/B for that test (same box, Release, solo, alternating, -V; harvest tgz 47f6eb8c): mean wall T3 aa30961 51.33 s vs T4 51.40 s (+0.14%); peak RSS at n=1,000 T3 19,941 KiB vs T4 19,987 KiB; at n=10,000 T3 37,147 KiB vs T4 37,149 KiB (+2 KiB, within the static delta of +512 B per Series<double>, about +10.8 KiB per host; nothing super-linear). A read-only trace (PERF) found no read-through cache consulted per input on that fixture: T4 adds two O(1) scope guards and every O(n)-per-input file is byte-identical to T3. The 160 s miss is load contention, not T4. The cache capacity is a parameter (strategy_set_cache_policy_v1; capacity 0 = off). Follow-up carded by CORE (CR-583 class): make test_native_live_report_memory's budget a parameter instead of a fixed 160 s.