Skip to content

Commit f05e632

Browse files
luisleo526claude
andcommitted
Merge the engine 1.11.1 train into the percent default quantity cap
The train now holds the changes merged into it so far, among them the fill-time drawdown rule (#804). The only textual conflict was the unreleased section of the changelog, where both lines are kept; the adapter, the host, the state hash and the test registrations merged without edits. Refs #805. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2 parents 09b0b15 + 1eab934 commit f05e632

31 files changed

Lines changed: 1446 additions & 250 deletions

‎CHANGELOG.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ version number follows semantic versioning over the surfaces the
88
## Unreleased
99

1010
- A percent-of-equity default quantity above 100 percent is now capped at 100 percent of equity when `margin_long` and `margin_short` are both 0 (#805).
11+
- `strategy.risk.max_drawdown` is now tested at the order fills that close units, on the equity at that fill with open positions included, against the highest equity at such fills (the equity at an opening fill or at a bar close does not raise it), and a breach closes every open position at that fill's price (exit text "Close Position (Max Drawdown)"), cancels the pending orders and blocks later entries; before, it was measured once per bar close and only refused new entries (#804).
1112
- The internal native Store live-object policy defaults to 0 (unbounded).
1213
The former capped default of 1,048,576 is now 0. Native limits are kept
1314
exactly: every positive uint64, including UINT64_MAX, is finite;
@@ -204,6 +205,16 @@ version number follows semantic versioning over the surfaces the
204205
occupant, and deleting the oldest drawing moves the directory's entries).
205206
The bound, the typed overflow outcome, the slot reuse order, the
206207
stale-handle behaviour and every value a script reads are unchanged.
208+
- The closed-trade read cache of the trade ledger places rows by count. While a
209+
run holds no more closed trades than the cache has entries (215,070 by
210+
default) each trade owns the slot of its ordinal: a read makes one probe, and
211+
the cache commits only the pages the trades reach (25,000 closed trades hold
212+
98 pages, 7.8 MB, where four-way sets hold 391, 31.2 MB). With more closed
213+
trades the four-way sets apply, and the cached rows are dropped when a run
214+
crosses the capacity in either direction. In a four-way set a hit on the most
215+
recent entry changes nothing and a hit on another entry ages only the entries
216+
more recent than it, so a full set replaces the entry touched longest ago. A
217+
miss is probed once. Every value a script reads is unchanged.
207218

208219
## 1.9.1 — 2026-10-31
209220

‎docs/pages/memory-management.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -351,7 +351,7 @@ in backing.
351351
| Store | Resident bound | Bytes per entry | Spills to scratch |
352352
| --- | --- | --- | --- |
353353
| Trade ledger (`TradeLedger`), descriptor window | 64 rows (`kResidentRows`, `include/pineforge/trade_ledger.hpp:48`), and a fixed payload tail of the same size | 168 B per row descriptor (`kRecordBytes`), so 10,752 B for the rows and 10,752 B for the tail; the four strings of a row are variable | Retiring sink runs erase published rows and return complete head chunks, keeping at most one head chunk per extent plus the unsealed suffix; each head retains fewer than 4,096 B of the newest retired index and payload bytes per file (about 24 trade descriptors) until the next zero-fill block completes or the chunk holding them is returned; other runs retain 168 B plus text per closed trade |
354-
| Trade ledger, read cache | Default cap: a provisional byte cap of 67,108,864 (`kDefaultLedgerCacheMaxBytes`, 64 MiB, not measurement-selected), which holds 215,070 occupied entries (840 full 256-slot pages and a clipped page of 30) with or without a report sink; an explicit row setting selects the finite row route instead (`kSinkCacheRows = kDefaultCacheRows` is its 65,536-entry legacy geometry; the row-setting ceiling `kMaxCacheRows` is 1,048,576). The ledger cap is policy-controlled: frozen host bytes for `PF_CACHE_KIND_LEDGER_CACHE` select the largest row count R with `R*kLedgerEntryBytes + ceil(R/256)*sizeof(void*) <= capacity_bytes`, including the private page directory. The host and standalone byte setters share `TradeLedger::cache_rows_for_capacity` (`include/pineforge/trade_ledger.hpp:80`); run begin applies the frozen policy through `CacheDefaults::apply_ledger_cache` (`src/native_execution_consumer.cpp:3227`), which gives an omitted setting `TradeLedger::set_ledger_cache_max_bytes` and an explicit one `TradeLedger::set_cache_rows` (`include/pineforge/cache_policy.hpp:119-124`). Zero bypasses; for an explicit host setting, nonzero values below one entry plus one pointer and values converting above the row ceiling are typed setting errors. Storage is private: a global ordinal chooses one four-way set (`global % ceil(cap/4)`, the last set partial when the cap is not a multiple of four) and a full set replaces the way touched longest ago; the first read creates `ceil(cap/256)` owning pointers, and the first touch of page `p` allocates `min(256,cap-256*p)` slots (`TradeLedger::cache_resident_bytes`, `include/pineforge/trade_ledger.hpp:166-172`). Page growth preserves entries through the 64/128/256 boundaries. Writes publish to backing before an optional cache fill. Retirement and a rewind invalidate every hot entry and release every optional page (the directory stays); the retained rows allocate their pages again by first touch, so allocated slots never exceed the cap. A directory or page allocation failure bypasses the cache: an explicit row setting stays bypassed for that ledger, while the byte default retries once the retained population has doubled since the failed attempt; reads, appends and replacements continue through backing. A copy of a ledger starts with a cold cache; a move hands over the whole cache, so the target is as hot as the source was and the source is left empty and cold with its declared policy | 312 B per slot (`kLedgerEntryBytes`: 168 B record, 16 B key and generation, 128 inline string bytes); up to 79,872 B per page, with the last page clipped to the cap. The charge `resident_cache_bytes(cap)` is `sizeof(TradeLedger)+sizeof(State)+cap*312+ceil(cap/256)*sizeof(void*)`: the default optional allowance is 67,108,864 B on LP64, of which the 215,070-entry default charges 67,108,568 B, comprising 67,101,840 entry bytes plus 6,728 table bytes (841 pointers); an explicit 65,536-entry cap charges 20,449,280 B, comprising 20,447,232 entry bytes plus 2,048 table bytes. The fixed base is separate. `cache_resident_bytes()` reports the fixed base, allocated directory and allocated pages once in cache resident bytes. The base includes the descriptor window, payload tail and zero-row read workspace; allocator overhead, returned owning Trades and explicit legacy materialization are excluded. Accounted allocation never exceeds the charge. The cap is independent of trade and bar counts | Not applicable (a cache over the retained rows) |
354+
| Trade ledger, read cache | Default cap: a provisional byte cap of 67,108,864 (`kDefaultLedgerCacheMaxBytes`, 64 MiB, not measurement-selected), which holds 215,070 occupied entries (840 full 256-slot pages and a clipped page of 30) with or without a report sink; an explicit row setting selects the finite row route instead (`kSinkCacheRows = kDefaultCacheRows` is its 65,536-entry legacy geometry; the row-setting ceiling `kMaxCacheRows` is 1,048,576). The ledger cap is policy-controlled: frozen host bytes for `PF_CACHE_KIND_LEDGER_CACHE` select the largest row count R with `R*kLedgerEntryBytes + ceil(R/256)*sizeof(void*) <= capacity_bytes`, including the private page directory. The host and standalone byte setters share `TradeLedger::cache_rows_for_capacity` (`include/pineforge/trade_ledger.hpp:80`); run begin applies the frozen policy through `CacheDefaults::apply_ledger_cache` (`src/native_execution_consumer.cpp:3227`), which gives an omitted setting `TradeLedger::set_ledger_cache_max_bytes` and an explicit one `TradeLedger::set_cache_rows` (`include/pineforge/cache_policy.hpp:119-124`). Zero bypasses; for an explicit host setting, nonzero values below one entry plus one pointer and values converting above the row ceiling are typed setting errors. Storage is private. While the ledger holds at most `cap` rows, row `n` owns slot `n` (one way), so the pages follow the rows (25,000 rows hold 98 pages, 7.8 MB, not the cap); once it holds more, a global ordinal chooses one four-way set (`global % ceil(cap/4)`, the last set partial when the cap is not a multiple of four) and a full set replaces the way touched longest ago. The change from one placement to the other, an append past `cap` or a rewind back to `cap` rows or fewer, drops every cached row and keeps the allocated pages. The first read creates `ceil(cap/256)` owning pointers, and the first touch of page `p` allocates `min(256,cap-256*p)` slots (`TradeLedger::cache_resident_bytes`, `include/pineforge/trade_ledger.hpp:166-172`). Page growth preserves entries through every page boundary (every 256 rows within the cap). Writes publish to backing before an optional cache fill. Retirement and a rewind invalidate every hot entry and release every optional page (the directory stays); the retained rows allocate their pages again by first touch, so allocated slots never exceed the cap. A directory or page allocation failure bypasses the cache: an explicit row setting stays bypassed for that ledger, while the byte default retries once the retained population has doubled since the failed attempt; reads, appends and replacements continue through backing. A copy of a ledger starts with a cold cache; a move hands over the whole cache, so the target is as hot as the source was and the source is left empty and cold with its declared policy | 312 B per slot (`kLedgerEntryBytes`: 168 B record, 16 B key and generation, 128 inline string bytes); up to 79,872 B per page, with the last page clipped to the cap. The charge `resident_cache_bytes(cap)` is `sizeof(TradeLedger)+sizeof(State)+cap*312+ceil(cap/256)*sizeof(void*)`: the default optional allowance is 67,108,864 B on LP64, of which the 215,070-entry default charges 67,108,568 B, comprising 67,101,840 entry bytes plus 6,728 table bytes (841 pointers); an explicit 65,536-entry cap charges 20,449,280 B, comprising 20,447,232 entry bytes plus 2,048 table bytes. The fixed base is separate. `cache_resident_bytes()` reports the fixed base, allocated directory and allocated pages once in cache resident bytes. The base includes the descriptor window, payload tail and zero-row read workspace; allocator overhead, returned owning Trades and explicit legacy materialization are excluded. Accounted allocation never exceeds the charge. The cap is independent of trade and bar counts | Not applicable (a cache over the retained rows) |
355355
| Command journal | 2,048 events (`JournalStore::kMaxResident`, `src/native_order.cpp:1806`), in 32-event chunks (`kChunk`); spilling stops once the resident count is at most 128 (`kKeepResident`, a stopping threshold rather than a floor) | Not a constant: an event is a variant with variable payload and `sizeof` of a resident slot is not stated in the source and was not compiled. Plus one lazy 163,840 B spill buffer per journal: a 32,768 B write buffer and four 32,768 B reload buffers (`SpillStorage`) | Older events, in pages of 32,768 B (`kSpillPageBytes`, `src/native_order.cpp:1827`) read back through four decoded page slots. An event whose serialized form exceeds one page is refused with `EFBIG` (about 1,360 external openings) and stays resident |
356356
| Order book | 4,096 rows | See the drain plateau below | Older rows |
357357
| Command-store resident prefix / read cache | Up to five 64 KiB payloads, each with at most 128 direct-mapped lines in `ReadThroughCache`. Default K is `5 * (65536 + 128 * (ReadThroughCache::line_bytes(512) - 512) + 2048)` bytes (`CachedScratchBacking::kCommandScratchMemoryBytes`, `src/command_scratch_cache.hpp`): the slab includes all shared tags, pins and replacement metadata; the 2 KiB allowance bounds each decorator's fixed control. For another representable build default C = `kCacheBytes`, the same bound is `K = 5 * (C + floor(C/512) * (ReadThroughCache::line_bytes(512) - 512) + 2048)`. Owners are trade, dependency, exit, lazy dormant book and lazy retained-sequence index. Pages larger than the selected payload (64 KiB by default) or smaller than 512 B bypass it | Independent of extent and bars. For internal payload capacity C and page size P, N = floor(C/P) and optional slab bytes are `C + N * (ReadThroughCache::line_bytes(P) - P)`; control bytes are separate. Capacities are `std::size_t` and page ids are 64-bit; no capacity-sized array enlarges the fixed object when the default is raised. Zero, sub-page or unrepresentable capacity bypasses. The host policy supplies independent inclusive allowances for the supported owners. Optional slab allocation failure selects direct backing I/O. The former 330 KiB K did not include the shared cache's larger metadata and no longer applies | Each disk store holds its prefix in the same slab until its extent exceeds the selected prefix capacity, then spills the prefix once and lends the slab to the shared read cache. Shared tags are constructed only after the entire prefix has landed. Failed spills retain the authoritative prefix for rollback and retry. Direct mode still charges logical extent against the same byte budget. Attached arenas retain direct mapped admission |
@@ -548,7 +548,7 @@ host add up the existing owners and workspaces; see
548548

549549
| Kind of store | Default budget | Resident memory at that budget |
550550
| --- | --- | --- |
551-
| Trade ledger | Descriptor window of 64 rows; private read cache defaults to a provisional byte cap of 67,108,864, holding up to 215,070 occupied entries, with or without a report sink. Hosts control its inclusive entry-and-directory byte cap through `PF_CACHE_KIND_LEDGER_CACHE`, within the existing 1,048,576-row setting ceiling; zero bypasses. Four-way set placement uses pages of at most 256 slots, clipped to the cap, allocated on first touch and all released when a retiring ledger seals or a rewind invalidates its hot entries. A directory or page allocation failure bypasses the cache (an explicit row setting for the ledger's lifetime, the byte default until the retained population has doubled since the failed attempt); backing reads and writes continue | 21,504 B for the window and its tail, plus a default optional allowance of 67,108,864 B on LP64, of which it charges 67,108,568 B (67,101,840 B of entries and a lazy 6,728 B pointer table); `resident_cache_bytes(rows)` at the default's 215,070 rows charges entries and table plus `sizeof(TradeLedger)+sizeof(State)` (which already includes the window, tail and zero-row read workspace). `cache_resident_bytes()` counts only allocated pages and the directory above that base, once; accounted allocation never exceeds the charge |
551+
| Trade ledger | Descriptor window of 64 rows; private read cache defaults to a provisional byte cap of 67,108,864, holding up to 215,070 occupied entries, with or without a report sink. Hosts control its inclusive entry-and-directory byte cap through `PF_CACHE_KIND_LEDGER_CACHE`, within the existing 1,048,576-row setting ceiling; zero bypasses. Placement (a slot per row up to the cap, four-way sets beyond it) uses pages of at most 256 slots, clipped to the cap, allocated on first touch and all released when a retiring ledger seals or a rewind invalidates its hot entries. A directory or page allocation failure bypasses the cache (an explicit row setting for the ledger's lifetime, the byte default until the retained population has doubled since the failed attempt); backing reads and writes continue | 21,504 B for the window and its tail, plus a default optional allowance of 67,108,864 B on LP64, of which it charges 67,108,568 B (67,101,840 B of entries and a lazy 6,728 B pointer table); `resident_cache_bytes(rows)` at the default's 215,070 rows charges entries and table plus `sizeof(TradeLedger)+sizeof(State)` (which already includes the window, tail and zero-row read workspace). `cache_resident_bytes()` counts only allocated pages and the directory above that base, once; accounted allocation never exceeds the charge |
552552
| Command journal | 2,048 resident events; spill stopping threshold 128 | Not a constant per event; plus 163,840 B of spill buffers per journal once it first spills |
553553
| Placement rows | No budget: the live rows plus the rows the retirement rules pin | Not measured |
554554
| Id tables | Cohort table: resident rows follow the spill rule above (at most twice the rows left by the last pass, plus 64, plus pinned rows), and the bounded profile admits at most 4,096 live cohorts. The ordered index is read through a 64 KiB window per scratch file | 64 KiB per file; the index itself is on disk |

‎include/pineforge/source/pine_adapter.hpp‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@ inline constexpr char kSourceAdapterDomain[] = "pineforge-source-adapter/v4";
6464
// layer supplies the corresponding report labels.
6565
inline constexpr char kMarginCallLabel[] = "__margin_call__";
6666
inline constexpr char kIntradayLossComment[] = "Close Position (Max intraday Loss)";
67+
inline constexpr char kMaxDrawdownComment[] = "Close Position (Max Drawdown)";
6768
// The published C contract matched this PREFIX, not the whole comment, so the
6869
// classifier keeps matching the prefix.
6970
inline constexpr char kFillCapCommentPrefix[] =
@@ -1573,6 +1574,7 @@ struct PineRiskState {
15731574
// its latches.
15741575
double observed_peak_equity = std::numeric_limits<double>::quiet_NaN();
15751576
double observed_max_drawdown = 0.0;
1577+
double fill_peak_equity = std::numeric_limits<double>::quiet_NaN();
15761578
std::int64_t intraday_block_day = std::numeric_limits<std::int64_t>::min();
15771579
bool intraday_cancel_pending = false;
15781580
};

‎include/pineforge/trade_ledger.hpp‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -158,11 +158,11 @@ class TradeLedger {
158158
static size_type resident_cache_bytes(size_type rows);
159159
// Actual allocation: fixed base + ceil(cache_rows()/256) owning pointers
160160
// after first read + min(256,cache_rows()-256*p)*kLedgerEntryBytes per page p.
161-
// Four-way sets (a final partial set is permitted) use bounded rank
162-
// replacement. Writes only fill an existing cache. No rebuild/coexisting
163-
// replacement allocation occurs. Retirement may release every hot page.
164-
// The fixed base includes the zero-row read workspace. The directory
165-
// and allocated entry pages each enter the common cache accounting once.
161+
// Placement follows the row count: up to cache_rows() rows, row n owns slot n (one way);
162+
// with more rows, four-way sets (a final partial set is permitted) use bounded rank
163+
// replacement. A crossing drops the cached rows, keeps the pages. Writes only fill an
164+
// existing cache; no rebuild or coexisting allocation. Retirement may release every hot page.
165+
// The fixed base (with the zero-row read workspace), directory and entry pages count once.
166166
size_type cache_resident_bytes() const;
167167
enum class CacheBypassReason : std::uint32_t { none, allocation_failed };
168168
struct CacheStats {

‎scripts/check_twin_parity.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@
2929
# executable code means a ledger/inventory rewrite cannot silently redefine
3030
# what the guard proves.
3131
BASE_MANIFEST_SHA256 = "a20f80eb29af455400fd2ab1bdef86e477af3d81891b1207623b4c890f6056db"
32-
INVENTORY_SHA256 = "04fd304bcc8f86a2abd4c121de88465307eaf069a8e5f1285307fdd45189b8b0"
33-
LEDGER_SHA256 = "c568999da8777ff967cdb3fcf007f382f88902bf78f9e02c2c5d7d15de22e9ba"
32+
INVENTORY_SHA256 = "a38305d7d402a6a236c53039ae4e4806caa88bbd6dfa3e554db01c26529e9a03"
33+
LEDGER_SHA256 = "5956795e5a65e80a79916e6ef0ced46dcc5474273fca1e5f166f09ed60e027d1"
3434

3535
APPENDIX_HEADING = "## Appendix 5 — CHECK-parity unobservable literal ledger"
3636
TABLE_HEADING = "| base file:line | CHECK text | reason unobservable | covering twin row |"

0 commit comments

Comments
 (0)