Skip to content

Commit b4a2c19

Browse files
authored
engine: let the command scratch cache size be raised past 64 KiB (#661)
Make the command-scratch cache size buildable at any size. **Problem** Changing `kCacheBytes` above its default did not compile: the per-backing slot table was an inline array sized from `kCacheBytes`, and `static_assert(sizeof(CachedScratchBacking) <= 2 * 1024)` failed (`command_scratch_cache.hpp:76`). **Change** - The slot table moves out of the object into one construction-time `new (std::nothrow)` allocation of `slot_count_` entries (the slots this backing actually uses), beside the page buffer. A failed allocation reports the same typed out-of-memory failure as the page buffer. - The `static_assert` now covers only the fixed part of the object (1 KiB). There is no upper cap on the cache size; sizes are `std::size_t` and page ids are 64-bit. - The memory constant still charges the worst case (`kMaxSlots` entries at 512-byte pages): 202,752 bytes at the default, unchanged. - Docs state the formula; the source-bounds inventory records the new allocation site. ### Large cache sizes The cache size is a plain `std::size_t` constant (64 KiB by default) and nothing in the class caps it: page ids are 64-bit, the slot table is sized from the constant and the page size, and the only `static_assert` covers the fixed part of the object. What was run, on Linux CI machines in a Release build, with only the one line that sets the constant changed: - Before this change, on main, raising the constant to 1 MiB does not build: the `static_assert` on the object size fails inside the library itself. - With the constant raised to 1 GiB and to 4 GiB the library and every test target build, and seven tests were run once each (results below). An earlier head of this change, built at 1 MiB, 16 MiB, 64 MiB and 256 MiB, compiled the library; its only compile errors were two default-size assertions in the new test, since made size-independent. - At 1 GiB and at 4 GiB, one run each of seven tests (the new slot-table test, the two allocation-failure tests, the summary-metrics allocation test, the command-store spill test, the report-sink curve test and the production file-backing test): - Pass at both sizes: the new slot-table test (19 s at 1 GiB, 88 s at 4 GiB), both allocation-failure tests, the summary-metrics allocation test and the spill test (28 s and 88 s). These are the product rows: they build a real cache, count its allocations and compare every read, hit and miss with an independent model. - Two tests do not pass at either size; both are limits of the test, not of the cache: - the report-sink curve test fails only above the default cache size: it asserts that the largest array of its run is the report curve, which holds only while the cache buffer is smaller than the curve (true at the 64 KiB default, where the test passes; the same assertion failed at 1 MiB to 256 MiB before the last edit of this change); - the production file-backing test fails identically on unchanged main on the same box: it needs real file media, and the scratch filesystem of these runs is an overlay that cannot provide it (see the control run). - At the default size, the full verification of this exact head (all eight steps) is the merge evidence; its result is posted on this pull request. Memory at a large constant: the slot table is allocated with the slot count of the page size of each store, so with 64 KiB pages and a 4 GiB constant it is 64 Ki entries, not the 8 Mi entries of the worst case (512-byte pages). The documented fixed bound still charges the worst case, 3 * (cache bytes + table at 512-byte pages + fixed object), which is 202,752 bytes at the default. This replaces #658: the same change rebuilt as one commit on the current main, so that its message names the documentation sentences it replaces. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- Commits on the PR head (verbatim): * 3188351 engine: let the command scratch cache size be raised past 64 KiB The command-store cache decorator kept its direct-mapped slot table inline (`std::array<ScratchPageId, kCacheBytes / 512>`), so the object grew with the cache constant and the size assertion failed at every cache size above 64 KiB (16,488 bytes at 1 MiB, about 4.2 MB at 256 MiB). The cache size could therefore not be raised and built. The table now lives in one allocation made at construction beside the page buffer, with the same typed out-of-memory failure, and the assertion states the fixed part of the object only. The table has one entry per slot of the store's own page size (`kCacheBytes / page size` entries), so a store with 64 KiB pages and a 4 GiB constant holds 64 Ki entries, not the 8 Mi entries of the worst case of 512-byte pages. The documented fixed bound still charges that worst case, as a formula of the constants: `3 * (kCacheBytes + kMaxSlots * sizeof(ScratchPageId) + kFixedObjectBytes)`, which is 202,752 bytes (198 KiB) at the default of 64 KiB. The slot function, admission rule, resident prefix, hit and miss behaviour and the bytes are unchanged; nothing in an I/O operation allocates or throws. The class lives in a source-only header, so no public header or ABI declaration changes. There is no upper cap on the constant: page ids and sizes are 64-bit, and the only limit is memory, reported as the typed out-of-memory failure. Tests: a new test drives a real decorator against an independent direct-mapped model at 512 B, 4 KiB, 64 KiB and 64 KiB + 1 pages (the last must bypass the cache) and asserts the bound formula at the default. The two existing allocation-census tests now count the slot table: the run-scratch allocation test derives the expected table size from each entry's page size (the exit entry opens its store with the platform page size), and the report-sink curve test counts the table in its array census. The source allocation-bound inventory lists the new site. Large cache sizes. With only the line that sets the constant changed, on Linux CI machines in a Release build: before this change, on main, a constant of 1 MiB does not build (the size assertion fails inside the library itself). With the constant raised to 1 GiB and to 4 GiB the library and every test target build, and seven tests were run once each (results below). An earlier head of this change, built at 1 MiB, 16 MiB, 64 MiB and 256 MiB, compiled the library; its only compile errors were two default-size assertions in the new test, since made size-independent. At 1 GiB and at 4 GiB, seven tests were run once each; the new slot-table test, both allocation-failure tests, the summary-metrics allocation test and the spill test pass at both sizes. Two do not, and both are limits of the test, not of the cache: - the report-sink curve test fails only above the default cache size: it asserts that the largest array of its run is the report curve, which holds only while the cache buffer is smaller than the curve; - the production file-backing test fails identically on unchanged main on the same box: it needs real file media, and the scratch filesystem of these runs is an overlay that cannot provide it. Replaced documentation sentences, named for the documentation revert check. This change replaces three sentences that state the fixed bound as a bare figure, all introduced by commit 8713a82 ("scratch: keep a run's command scratch resident up to a fixed size; sync the report sink on"). Each is replaced by the same bound written as a formula of the cache constants, which is still 202,752 bytes at the default: - docs/design/arena-bound.md: "The command-scratch resident/cache ledger term is fixed **K = 202,752 bytes/run**:" - docs/design/arena-bound.md: "`3 * (65536 + 2048)`, named at `src/command_scratch_cache.hpp:N`, including three 64-KiB buffers and their decorator/slot-key objects (`static_assert(`" - docs/pages/memory-management.md: "Fixed K = 202,752 B (198 KiB) per run: three 64 KiB page buffers plus 2 KiB per decorator, including slot keys (`CachedScratchBacking::kCommandScratch" Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
1 parent 18f55bb commit b4a2c19

10 files changed

Lines changed: 463 additions & 45 deletions

‎ci/cr150-source-bounds.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1354,7 +1354,7 @@ entries:
13541354
- {id: "U1313", file: "include/pineforge/trade_ledger.hpp", symbol: "exit_comment_at", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 152}
13551355
- {id: "U1314", file: "include/pineforge/trade_ledger.hpp", symbol: "exit_id_at", ops: ["decl"], container: "std::string", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 153}
13561356
- {id: "U1522", file: "src/command_scratch_cache.cpp", symbol: "budget_", ops: ["grow"], container: "unresolved by the index: budget_.reserve(...)", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 94}
1357-
- {id: "U1316", file: "src/command_scratch_cache.cpp", symbol: "reset", ops: ["alloc"], container: "unresolved by the index: new ", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 25}
1357+
- {id: "U1316", file: "src/command_scratch_cache.cpp", symbol: "reset", ops: ["alloc"], container: "unresolved by the index: new ", owner: "engine-core", phase: "unclassified", state: "unreviewed", note: "Two construction-time allocations of the command scratch cache decorator, both made once in the constructor when the page size is cacheable: the page buffer (kCacheBytes) and the direct-mapped slot table (kMaxSlots entries of ScratchPageId, zero-initialised, then set to the no-page id). Both are owned by the object and freed with it; neither depends on bars, trades or extent; a failure of either is the typed out_of_memory failure. The fixed bound counts both: K = 3 * (kCacheBytes + kMaxSlots * sizeof(ScratchPageId) + kFixedObjectBytes). Unsupported member-array admission stays OPEN.", line: 25}
13581358
- {id: "U1523", file: "src/command_scratch_cache.cpp", symbol: "resident_", ops: ["assign"], container: "unresolved by the index: resident_ = ...", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 126}
13591359
- {id: "U1317", file: "src/command_scratch_production.cpp", symbol: "budget", ops: ["alloc"], container: "unresolved by the index: make_shared", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 390}
13601360
- {id: "U1318", file: "src/command_scratch_production.cpp", symbol: "budget_", ops: ["grow"], container: "unresolved by the index: budget_.reserve(...)", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 175}

‎ci/cr150-source-sites.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13174,6 +13174,18 @@
1317413174
"spelling": "new",
1317513175
"ordinal": 1
1317613176
},
13177+
{
13178+
"file": "src/command_scratch_cache.cpp",
13179+
"symbol": "reset",
13180+
"scope": [
13181+
"namespace pineforge :: detail #1",
13182+
"CachedScratchBacking :: CachedScratchBacking ( std :: unique_ptr < ScratchBacking > inner , ScratchIoObserver observer , std :: size_t resident_bytes , std :: shared_ptr < CommandScratchByteBudget > budget ) : ScratchBacking ( inner ? inner -> kind ( ) : ScratchBackingKind :: disk , inner ? inner -> page_size ( ) : 0 , observer ) , inner_ ( std :: move ( inner ) ) , slot_count_ ( page_size ( ) > = kMinPageBytes && page_size ( ) < = kCacheBytes ? kCacheBytes / page_size ( ) : 0 ) , resident_pages_ ( slot_count_ ! = 0 ? std :: min ( resident_bytes , kCacheBytes ) / page_size ( ) : 0 ) , resident_ ( resident_pages_ ! = 0 && inner_ -> page_count ( ) = = 0 ) , budget_ ( std :: move ( budget ) ) #1",
13183+
"if ( slot_count_ ! = 0 ) #1"
13184+
],
13185+
"cls": "alloc",
13186+
"spelling": "new",
13187+
"ordinal": 2
13188+
},
1317713189
{
1317813190
"file": "src/command_scratch_cache.cpp",
1317913191
"symbol": "budget_",

‎docs/design/arena-bound.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -692,11 +692,11 @@ the resident-cap change changes no per-user storage term, command share, closed
692692
The unbound journal's `ExtFile` uses `ScratchFile`, which can claim arena regions; earlier spill
693693
does not certify its lifetime extent. The bound codec's existing command share supplies no private RSS/AS certification.
694694

695-
The command-scratch resident/cache ledger term is fixed **K = 202,752 bytes/run**:
696-
`3 * (65536 + 2048)`, named at `src/command_scratch_cache.hpp:28`, including three
697-
64-KiB buffers and their decorator/slot-key objects (`static_assert(sizeof(CachedScratchBacking)` at `src/command_scratch_cache.hpp:76`).
695+
The command-scratch resident/cache ledger term is fixed **K = `3 * (kCacheBytes + kMaxSlots * sizeof(ScratchPageId) + kFixedObjectBytes)`**:
696+
at the default this is **202,752 bytes/run**, `3 * (65536 + 1024 + 1024)`, named at
697+
`src/command_scratch_cache.hpp:28`, including three 64-KiB buffers, slot tables allocated with each wrapped backing's page slot count (K is charged at the worst case of 512-byte pages) and fixed objects (`static_assert(sizeof(CachedScratchBacking)` at `src/command_scratch_cache.hpp:76`).
698698
Each disk store reuses one buffer for its resident prefix, then its read cache;
699-
crossing 64 KiB spills once. Attached arenas retain direct mapped admission.
699+
crossing `kCacheBytes` (64 KiB at default) spills once. Attached arenas retain direct mapped admission.
700700
Charge K (prose ledger; scripts/arena_bound.py has no K input) alongside existing workspaces/inner backings, replacing the old read-cache allowance; K changes no arena-storage formula or certification.
701701

702702
CORE-TOP's A5-WIDER brief reports a **44.4–46.2 MB pre-flat-port resident

‎docs/pages/memory-management.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -321,7 +321,7 @@ store; the part that spills grows in the scratch backing.
321321
| Trade ledger, read cache | Up to 65,536 occupied entries with or without a report sink (`kSinkCacheRows = kDefaultCacheRows`; the ceiling `kMaxCacheRows` is 1,048,576). Direct-mapped by global index modulo the fixed cap; the first read creates a table of `ceil(cap/256)` owning pointers, and each first-touched page `p` allocates `min(256,cap-256*p)` slots (the comment on `TradeLedger::cache_resident_bytes`, `include/pineforge/trade_ledger.hpp:105-109`). Page growth preserves existing entries; writes fill only an existing cache. Retirement frees pages outside the retained window, bounding allocated slots by `min(cap,(ceil(retained_rows/256)+2)*256)`. Chosen by the engine when the run begins, through `TradeLedger::set_cache_rows` (`src/native_execution_consumer.cpp:3192-3193`); the C API has no setter for it in this release | 312 B per slot (`kLedgerEntryBytes`: 168 B record, 16 B key and generation, 128 inline string bytes); up to 79,872 B per page, with the last page clipped to the cap. The charge `resident_cache_bytes(cap)` is `sizeof(TradeLedger)+sizeof(State)+cap*312+ceil(cap/256)*sizeof(void*)`: at the default cap, 20,447,232 entry bytes (19.5 MiB) plus 2,048 table bytes on 64-bit targets and the fixed base. Actual allocation never exceeds this charge. Both caps are independent of trade and bar counts | Not applicable (a cache over the retained rows) |
322322
| Command journal | 2,048 events (`kMaxResident`, `src/native_order.cpp:1717`), in 32-event chunks (`kChunk`), with a floor of 128 events kept (`kKeepResident`) | Not a constant: an event is a variant with variable payload and `sizeof` of a resident slot is not stated in the source and was not compiled. Plus one lazy 163,840 B spill buffer per journal: a 32,768 B write buffer and four 32,768 B reload buffers (`SpillStorage`) | Older events, in pages of 32,768 B (`kSpillPageBytes`, `:1738`) read back through four decoded page slots. An event whose serialized form exceeds one page is refused with `EFBIG` (about 1,360 external openings) and stays resident |
323323
| Order book | 4,096 rows | See the drain plateau below | Older rows |
324-
| Command-store resident prefix / read cache | Fixed K = 202,752 B (198 KiB) per run: three 64 KiB page buffers plus 2 KiB per decorator, including slot keys (`CachedScratchBacking::kCommandScratchMemoryBytes`, `src/command_scratch_cache.hpp:28`; `static_assert(sizeof(CachedScratchBacking)` at `src/command_scratch_cache.hpp:76`); pages larger than 64 KiB or smaller than 512 B bypass the buffer | Counted in bytes; independent of extent and bars; replaces the previous read-cache allowance | Each disk store holds its prefix in the same buffer until its extent exceeds 64 KiB, then spills the prefix once and reuses the buffer as a read cache. Attached arenas retain direct mapped admission |
324+
| Command-store resident prefix / read cache | Fixed K = `3 * (kCacheBytes + kMaxSlots * sizeof(ScratchPageId) + kFixedObjectBytes)` = 202,752 B (198 KiB) per run at the default: three 64 KiB page buffers, a table allocated with the slot count for the wrapped backing's page size (K is charged at the worst case of 512-byte pages: 128 eight-byte slot keys, 1 KiB), and a 1 KiB fixed object (`CachedScratchBacking::kCommandScratchMemoryBytes`, `src/command_scratch_cache.hpp:28`; `static_assert(sizeof(CachedScratchBacking)` at `src/command_scratch_cache.hpp:76`); pages larger than `kCacheBytes` (64 KiB at default) or smaller than 512 B bypass the buffer; the constant has no upper cap in the class (at 4 GiB the table is 8 Mi entries of 8 B, 64 MiB per store) | Counted in bytes; independent of extent and bars; replaces the previous read-cache allowance | Each disk store holds its prefix in the same buffer until its extent exceeds `kCacheBytes` (64 KiB at default), then spills the prefix once and reuses the buffer as a read cache. Attached arenas retain direct mapped admission |
325325
| Id-table read caches | A separate fixed 192 KiB of page bytes **per adapter, not per engine** (`IdTableReadCache::kAdapterCacheBytes`, `include/pineforge/detail/id_table_read_cache.hpp`): only the adapter opts in its cohort AVL and exit-rank AVL at 64 KiB each, and erased minima and consumed cycles at 32 KiB each (data/index sharing one allocation per pair store). Other table instances default to zero capacity and allocate nothing. Lines are 4 KiB; allocation is lazy, falls back to direct reads on allocation failure, and never grows. All slab and arena paths bypass allocation and cache lookup, including unjailed regular-file arenas and inherited sealed memfds | Plus a fixed cache control object per table instance, each with sixteen page/file tags, one owning pointer and a slot count (`sizeof(IdTableReadCache)`); no per-id metadata. One adapter owns each enabled cache; no sharing across threads. Every write still reaches the file immediately; overlapping writes invalidate before I/O, appends invalidate from the old tail page onward, and index growth, clear, moves and backend changes invalidate all lines. Bound from constants, not a measurement | Not applicable (copy-in/copy-out reads over private disk files; formats and logical folds unchanged) |
326326
| Scratch files read through a bounded window | one 64 KiB window per file that enables it (`ScratchFile::kCacheBytes`, `include/pineforge/detail/scratch_file.hpp:40`) | Counted in bytes. It bounds the operating-system page cache the file pins (the kernel is asked to drop clean pages); it is not an engine heap cache | Not applicable |
327327
| Report sink writer | a fixed 64 KiB buffer; a record line may not exceed 1 MiB | Counted in bytes | The sink file or stream |

‎src/command_scratch_cache.cpp‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,11 @@ CachedScratchBacking::CachedScratchBacking(std::unique_ptr<ScratchBacking> inner
3636
bytes_.reset(new (std::nothrow) std::byte[kCacheBytes]);
3737
if (!bytes_)
3838
throw scratch_out_of_memory_failure();
39+
page_ids_.reset(new (std::nothrow) ScratchPageId[slot_count_]());
40+
if (!page_ids_)
41+
throw scratch_out_of_memory_failure();
42+
std::fill_n(page_ids_.get(), slot_count_, scratch_no_page);
3943
}
40-
page_ids_.fill(scratch_no_page);
4144
}
4245

4346
CachedScratchBacking::~CachedScratchBacking() noexcept {

‎src/command_scratch_cache.hpp‎

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -13,21 +13,21 @@ struct CommandScratchByteBudget;
1313
// Exclusive resident/read-through decorator for any ScratchBacking. All access to the
1414
// owned inner backing must go through this object; construct the inner with
1515
// NO observer and install the logical I/O observer here, including cache hits.
16-
// One fixed allocation serves first as a dense resident prefix, then as the
17-
// read cache after spill. At most three stores per run share the fixed bound K
18-
// below, including their decorator objects/slot keys. Nothing depends on extent
19-
// or bars. No I/O operation allocates or throws.
16+
// One fixed buffer serves first as a dense resident prefix, then as the
17+
// read cache after spill. Its slot table uses the wrapped backing's slot count.
18+
// K below charges the worst case (512-byte pages) for three stores, including
19+
// decorator objects/slot keys. Nothing depends on extent or bars; no I/O operation allocates or throws.
2020
//
2121
// Page size is immutable per backing. Direct-mapped keys therefore identify
2222
// pages in this one geometry (512 B through 1 MiB, including non-powers of
23-
// two); pages above 64 KiB bypass the cache, as do sizes below 512 B.
23+
// two); pages above kCacheBytes bypass the cache, as do sizes below 512 B.
2424
class CachedScratchBacking final : public ScratchBacking {
2525
public:
2626
static constexpr std::size_t kCacheBytes = 64 * 1024;
27-
// K = 198 KiB/run: three 64-KiB buffers plus 2 KiB of object/slot keys each.
28-
static constexpr std::size_t kCommandScratchMemoryBytes = 3 * (kCacheBytes + 2 * 1024);
29-
30-
// The inner must be nonnull. A page-buffer allocation failure is a typed
27+
static constexpr std::size_t kMinPageBytes = 512, kFixedObjectBytes = 1024, kMaxSlots = kCacheBytes / kMinPageBytes;
28+
static constexpr std::size_t kCommandScratchMemoryBytes =
29+
3 * (kCacheBytes + kMaxSlots * sizeof(ScratchPageId) + kFixedObjectBytes); // K charges three buffers, 512-byte-page tables and fixed objects.
30+
// The inner must be nonnull. A page-buffer or slot-table allocation failure is a typed
3131
// out_of_memory failure carried by the shared, prebuilt scratch exception.
3232
// resident_bytes is a construction-only cap, clamped to kCacheBytes;
3333
// zero retains the all-file/write-through mode. Only a fresh inner can
@@ -56,8 +56,8 @@ class CachedScratchBacking final : public ScratchBacking {
5656
ScratchStatus do_sync(ScratchDurability durability) noexcept override;
5757

5858
private:
59-
static constexpr std::size_t kMinPageBytes = 512;
60-
static constexpr std::size_t kMaxSlots = kCacheBytes / kMinPageBytes;
59+
// Cache geometry constants are public so the fixed bound names each component.
60+
// The remaining members carry the backing and cache state.
6161

6262
std::size_t slot(ScratchPageId id) const noexcept;
6363
void remember(ScratchPageId id, const std::byte* bytes) const noexcept;
@@ -70,10 +70,10 @@ class CachedScratchBacking final : public ScratchBacking {
7070
std::uint64_t resident_extent_ = 0;
7171
std::shared_ptr<CommandScratchByteBudget> budget_;
7272
std::unique_ptr<std::byte[]> bytes_;
73-
mutable std::array<ScratchPageId, kMaxSlots> page_ids_{};
73+
mutable std::unique_ptr<ScratchPageId[]> page_ids_;
7474
};
7575

76-
static_assert(sizeof(CachedScratchBacking) <= 2 * 1024,
77-
"three backing objects and page buffers must fit the fixed K");
76+
static_assert(sizeof(CachedScratchBacking) <= CachedScratchBacking::kFixedObjectBytes,
77+
"the fixed cache object must fit its fixed K component");
7878

7979
} // namespace pineforge::detail
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
list(APPEND TEST_SOURCES test_command_scratch_cache_slot_table)
2+
pineforge_add_unit_test(test_command_scratch_cache_slot_table)
3+
set_tests_properties(test_command_scratch_cache_slot_table PROPERTIES LABELS "fast;scratch")

0 commit comments

Comments
 (0)