You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit f05e632
Browse filesBrowse the repository at this point in the historyBrowse files
Merge the engine 1.11.1 train into the percent default quantity cap
The train now holds the changes merged into it so far, among them the
fill-time drawdown rule (#804). The only textual conflict was the unreleased
section of the changelog, where both lines are kept; the adapter, the host,
the state hash and the test registrations merged without edits.
Refs #805.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Copy file name to clipboardExpand all lines: CHANGELOG.md
+11Lines changed: 11 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,6 +8,7 @@ version number follows semantic versioning over the surfaces the
8
8
## Unreleased
9
9
10
10
- A percent-of-equity default quantity above 100 percent is now capped at 100 percent of equity when `margin_long` and `margin_short` are both 0 (#805).
11
+
-`strategy.risk.max_drawdown` is now tested at the order fills that close units, on the equity at that fill with open positions included, against the highest equity at such fills (the equity at an opening fill or at a bar close does not raise it), and a breach closes every open position at that fill's price (exit text "Close Position (Max Drawdown)"), cancels the pending orders and blocks later entries; before, it was measured once per bar close and only refused new entries (#804).
11
12
- The internal native Store live-object policy defaults to 0 (unbounded).
12
13
The former capped default of 1,048,576 is now 0. Native limits are kept
13
14
exactly: every positive uint64, including UINT64_MAX, is finite;
@@ -204,6 +205,16 @@ version number follows semantic versioning over the surfaces the
204
205
occupant, and deleting the oldest drawing moves the directory's entries).
205
206
The bound, the typed overflow outcome, the slot reuse order, the
206
207
stale-handle behaviour and every value a script reads are unchanged.
208
+
- The closed-trade read cache of the trade ledger places rows by count. While a
209
+
run holds no more closed trades than the cache has entries (215,070 by
210
+
default) each trade owns the slot of its ordinal: a read makes one probe, and
211
+
the cache commits only the pages the trades reach (25,000 closed trades hold
212
+
98 pages, 7.8 MB, where four-way sets hold 391, 31.2 MB). With more closed
213
+
trades the four-way sets apply, and the cached rows are dropped when a run
214
+
crosses the capacity in either direction. In a four-way set a hit on the most
215
+
recent entry changes nothing and a hit on another entry ages only the entries
216
+
more recent than it, so a full set replaces the entry touched longest ago. A
217
+
miss is probed once. Every value a script reads is unchanged.
Copy file name to clipboardExpand all lines: docs/pages/memory-management.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -351,7 +351,7 @@ in backing.
351
351
| Store | Resident bound | Bytes per entry | Spills to scratch |
352
352
| --- | --- | --- | --- |
353
353
| Trade ledger (`TradeLedger`), descriptor window | 64 rows (`kResidentRows`, `include/pineforge/trade_ledger.hpp:48`), and a fixed payload tail of the same size | 168 B per row descriptor (`kRecordBytes`), so 10,752 B for the rows and 10,752 B for the tail; the four strings of a row are variable | Retiring sink runs erase published rows and return complete head chunks, keeping at most one head chunk per extent plus the unsealed suffix; each head retains fewer than 4,096 B of the newest retired index and payload bytes per file (about 24 trade descriptors) until the next zero-fill block completes or the chunk holding them is returned; other runs retain 168 B plus text per closed trade |
354
-
| Trade ledger, read cache | Default cap: a provisional byte cap of 67,108,864 (`kDefaultLedgerCacheMaxBytes`, 64 MiB, not measurement-selected), which holds 215,070 occupied entries (840 full 256-slot pages and a clipped page of 30) with or without a report sink; an explicit row setting selects the finite row route instead (`kSinkCacheRows = kDefaultCacheRows` is its 65,536-entry legacy geometry; the row-setting ceiling `kMaxCacheRows` is 1,048,576). The ledger cap is policy-controlled: frozen host bytes for `PF_CACHE_KIND_LEDGER_CACHE` select the largest row count R with `R*kLedgerEntryBytes + ceil(R/256)*sizeof(void*) <= capacity_bytes`, including the private page directory. The host and standalone byte setters share `TradeLedger::cache_rows_for_capacity` (`include/pineforge/trade_ledger.hpp:80`); run begin applies the frozen policy through `CacheDefaults::apply_ledger_cache` (`src/native_execution_consumer.cpp:3227`), which gives an omitted setting `TradeLedger::set_ledger_cache_max_bytes` and an explicit one `TradeLedger::set_cache_rows` (`include/pineforge/cache_policy.hpp:119-124`). Zero bypasses; for an explicit host setting, nonzero values below one entry plus one pointer and values converting above the row ceiling are typed setting errors. Storage is private: a global ordinal chooses one four-way set (`global % ceil(cap/4)`, the last set partial when the cap is not a multiple of four) and a full set replaces the way touched longest ago; the first read creates `ceil(cap/256)` owning pointers, and the first touch of page `p` allocates `min(256,cap-256*p)` slots (`TradeLedger::cache_resident_bytes`, `include/pineforge/trade_ledger.hpp:166-172`). Page growth preserves entries through the 64/128/256 boundaries. Writes publish to backing before an optional cache fill. Retirement and a rewind invalidate every hot entry and release every optional page (the directory stays); the retained rows allocate their pages again by first touch, so allocated slots never exceed the cap. A directory or page allocation failure bypasses the cache: an explicit row setting stays bypassed for that ledger, while the byte default retries once the retained population has doubled since the failed attempt; reads, appends and replacements continue through backing. A copy of a ledger starts with a cold cache; a move hands over the whole cache, so the target is as hot as the source was and the source is left empty and cold with its declared policy | 312 B per slot (`kLedgerEntryBytes`: 168 B record, 16 B key and generation, 128 inline string bytes); up to 79,872 B per page, with the last page clipped to the cap. The charge `resident_cache_bytes(cap)` is `sizeof(TradeLedger)+sizeof(State)+cap*312+ceil(cap/256)*sizeof(void*)`: the default optional allowance is 67,108,864 B on LP64, of which the 215,070-entry default charges 67,108,568 B, comprising 67,101,840 entry bytes plus 6,728 table bytes (841 pointers); an explicit 65,536-entry cap charges 20,449,280 B, comprising 20,447,232 entry bytes plus 2,048 table bytes. The fixed base is separate. `cache_resident_bytes()` reports the fixed base, allocated directory and allocated pages once in cache resident bytes. The base includes the descriptor window, payload tail and zero-row read workspace; allocator overhead, returned owning Trades and explicit legacy materialization are excluded. Accounted allocation never exceeds the charge. The cap is independent of trade and bar counts | Not applicable (a cache over the retained rows) |
354
+
| Trade ledger, read cache | Default cap: a provisional byte cap of 67,108,864 (`kDefaultLedgerCacheMaxBytes`, 64 MiB, not measurement-selected), which holds 215,070 occupied entries (840 full 256-slot pages and a clipped page of 30) with or without a report sink; an explicit row setting selects the finite row route instead (`kSinkCacheRows = kDefaultCacheRows` is its 65,536-entry legacy geometry; the row-setting ceiling `kMaxCacheRows` is 1,048,576). The ledger cap is policy-controlled: frozen host bytes for `PF_CACHE_KIND_LEDGER_CACHE` select the largest row count R with `R*kLedgerEntryBytes + ceil(R/256)*sizeof(void*) <= capacity_bytes`, including the private page directory. The host and standalone byte setters share `TradeLedger::cache_rows_for_capacity` (`include/pineforge/trade_ledger.hpp:80`); run begin applies the frozen policy through `CacheDefaults::apply_ledger_cache` (`src/native_execution_consumer.cpp:3227`), which gives an omitted setting `TradeLedger::set_ledger_cache_max_bytes` and an explicit one `TradeLedger::set_cache_rows` (`include/pineforge/cache_policy.hpp:119-124`). Zero bypasses; for an explicit host setting, nonzero values below one entry plus one pointer and values converting above the row ceiling are typed setting errors. Storage is private. While the ledger holds at most `cap` rows, row `n` owns slot `n` (one way), so the pages follow the rows (25,000 rows hold 98 pages, 7.8 MB, not the cap); once it holds more, a global ordinal chooses one four-way set (`global % ceil(cap/4)`, the last set partial when the cap is not a multiple of four) and a full set replaces the way touched longest ago. The change from one placement to the other, an append past `cap` or a rewind back to `cap` rows or fewer, drops every cached row and keeps the allocated pages. The first read creates `ceil(cap/256)` owning pointers, and the first touch of page `p` allocates `min(256,cap-256*p)` slots (`TradeLedger::cache_resident_bytes`, `include/pineforge/trade_ledger.hpp:166-172`). Page growth preserves entries through every page boundary (every 256 rows within the cap). Writes publish to backing before an optional cache fill. Retirement and a rewind invalidate every hot entry and release every optional page (the directory stays); the retained rows allocate their pages again by first touch, so allocated slots never exceed the cap. A directory or page allocation failure bypasses the cache: an explicit row setting stays bypassed for that ledger, while the byte default retries once the retained population has doubled since the failed attempt; reads, appends and replacements continue through backing. A copy of a ledger starts with a cold cache; a move hands over the whole cache, so the target is as hot as the source was and the source is left empty and cold with its declared policy | 312 B per slot (`kLedgerEntryBytes`: 168 B record, 16 B key and generation, 128 inline string bytes); up to 79,872 B per page, with the last page clipped to the cap. The charge `resident_cache_bytes(cap)` is `sizeof(TradeLedger)+sizeof(State)+cap*312+ceil(cap/256)*sizeof(void*)`: the default optional allowance is 67,108,864 B on LP64, of which the 215,070-entry default charges 67,108,568 B, comprising 67,101,840 entry bytes plus 6,728 table bytes (841 pointers); an explicit 65,536-entry cap charges 20,449,280 B, comprising 20,447,232 entry bytes plus 2,048 table bytes. The fixed base is separate. `cache_resident_bytes()` reports the fixed base, allocated directory and allocated pages once in cache resident bytes. The base includes the descriptor window, payload tail and zero-row read workspace; allocator overhead, returned owning Trades and explicit legacy materialization are excluded. Accounted allocation never exceeds the charge. The cap is independent of trade and bar counts | Not applicable (a cache over the retained rows) |
355
355
| Command journal | 2,048 events (`JournalStore::kMaxResident`, `src/native_order.cpp:1806`), in 32-event chunks (`kChunk`); spilling stops once the resident count is at most 128 (`kKeepResident`, a stopping threshold rather than a floor) | Not a constant: an event is a variant with variable payload and `sizeof` of a resident slot is not stated in the source and was not compiled. Plus one lazy 163,840 B spill buffer per journal: a 32,768 B write buffer and four 32,768 B reload buffers (`SpillStorage`) | Older events, in pages of 32,768 B (`kSpillPageBytes`, `src/native_order.cpp:1827`) read back through four decoded page slots. An event whose serialized form exceeds one page is refused with `EFBIG` (about 1,360 external openings) and stays resident |
356
356
| Order book | 4,096 rows | See the drain plateau below | Older rows |
357
357
| Command-store resident prefix / read cache | Up to five 64 KiB payloads, each with at most 128 direct-mapped lines in `ReadThroughCache`. Default K is `5 * (65536 + 128 * (ReadThroughCache::line_bytes(512) - 512) + 2048)` bytes (`CachedScratchBacking::kCommandScratchMemoryBytes`, `src/command_scratch_cache.hpp`): the slab includes all shared tags, pins and replacement metadata; the 2 KiB allowance bounds each decorator's fixed control. For another representable build default C = `kCacheBytes`, the same bound is `K = 5 * (C + floor(C/512) * (ReadThroughCache::line_bytes(512) - 512) + 2048)`. Owners are trade, dependency, exit, lazy dormant book and lazy retained-sequence index. Pages larger than the selected payload (64 KiB by default) or smaller than 512 B bypass it | Independent of extent and bars. For internal payload capacity C and page size P, N = floor(C/P) and optional slab bytes are `C + N * (ReadThroughCache::line_bytes(P) - P)`; control bytes are separate. Capacities are `std::size_t` and page ids are 64-bit; no capacity-sized array enlarges the fixed object when the default is raised. Zero, sub-page or unrepresentable capacity bypasses. The host policy supplies independent inclusive allowances for the supported owners. Optional slab allocation failure selects direct backing I/O. The former 330 KiB K did not include the shared cache's larger metadata and no longer applies | Each disk store holds its prefix in the same slab until its extent exceeds the selected prefix capacity, then spills the prefix once and lends the slab to the shared read cache. Shared tags are constructed only after the entire prefix has landed. Failed spills retain the authoritative prefix for rollback and retry. Direct mode still charges logical extent against the same byte budget. Attached arenas retain direct mapped admission |
@@ -548,7 +548,7 @@ host add up the existing owners and workspaces; see
548
548
549
549
| Kind of store | Default budget | Resident memory at that budget |
550
550
| --- | --- | --- |
551
-
| Trade ledger | Descriptor window of 64 rows; private read cache defaults to a provisional byte cap of 67,108,864, holding up to 215,070 occupied entries, with or without a report sink. Hosts control its inclusive entry-and-directory byte cap through `PF_CACHE_KIND_LEDGER_CACHE`, within the existing 1,048,576-row setting ceiling; zero bypasses. Four-way set placement uses pages of at most 256 slots, clipped to the cap, allocated on first touch and all released when a retiring ledger seals or a rewind invalidates its hot entries. A directory or page allocation failure bypasses the cache (an explicit row setting for the ledger's lifetime, the byte default until the retained population has doubled since the failed attempt); backing reads and writes continue | 21,504 B for the window and its tail, plus a default optional allowance of 67,108,864 B on LP64, of which it charges 67,108,568 B (67,101,840 B of entries and a lazy 6,728 B pointer table); `resident_cache_bytes(rows)` at the default's 215,070 rows charges entries and table plus `sizeof(TradeLedger)+sizeof(State)` (which already includes the window, tail and zero-row read workspace). `cache_resident_bytes()` counts only allocated pages and the directory above that base, once; accounted allocation never exceeds the charge |
551
+
| Trade ledger | Descriptor window of 64 rows; private read cache defaults to a provisional byte cap of 67,108,864, holding up to 215,070 occupied entries, with or without a report sink. Hosts control its inclusive entry-and-directory byte cap through `PF_CACHE_KIND_LEDGER_CACHE`, within the existing 1,048,576-row setting ceiling; zero bypasses. Placement (a slot per row up to the cap, four-way sets beyond it) uses pages of at most 256 slots, clipped to the cap, allocated on first touch and all released when a retiring ledger seals or a rewind invalidates its hot entries. A directory or page allocation failure bypasses the cache (an explicit row setting for the ledger's lifetime, the byte default until the retained population has doubled since the failed attempt); backing reads and writes continue | 21,504 B for the window and its tail, plus a default optional allowance of 67,108,864 B on LP64, of which it charges 67,108,568 B (67,101,840 B of entries and a lazy 6,728 B pointer table); `resident_cache_bytes(rows)` at the default's 215,070 rows charges entries and table plus `sizeof(TradeLedger)+sizeof(State)` (which already includes the window, tail and zero-row read workspace). `cache_resident_bytes()` counts only allocated pages and the directory above that base, once; accounted allocation never exceeds the charge |
552
552
| Command journal | 2,048 resident events; spill stopping threshold 128 | Not a constant per event; plus 163,840 B of spill buffers per journal once it first spills |
553
553
| Placement rows | No budget: the live rows plus the rows the retirement rules pin | Not measured |
554
554
| Id tables | Cohort table: resident rows follow the spill rule above (at most twice the rows left by the last pass, plus 64, plus pinned rows), and the bounded profile admits at most 4,096 live cohorts. The ordered index is read through a 64 KiB window per scratch file | 64 KiB per file; the index itself is on disk |
0 commit comments