Skip to content

docs: add the memory-management page - #531

Merged
luisleo526 merged 1 commit into
mainfrom
docs/memory-management-170
Oct 26, 2026
Merged

luisleo526 merged 1 commit into
mainfrom
docs/memory-management-170

Conversation

@luisleo526

Copy link
Copy Markdown
Collaborator

What

Adds docs/pages/memory-management.md and links it from the docs index next to "Scratch tables".

The page answers one question: how much memory can a run use? It lists the controls (the environment variables and run_json.py flags, with their defaults and units), the fixed resident bounds, the scratch formula with a worked example, and what grows with bars, trades and open orders. Every constant and source citation on the page was checked against the source. Docs only: no code and no behaviour change.

Checks

  • lab remote preflight + docs on a clean tree: preflight passed (59 stages), docs build passed. The doc-anchors gate accepts every citation on the page.
  • The page was read statically by a second reviewer, who recomputed every number on it.

Notes for the release

The cohort-lookup row and the report-key rows describe the tree at the time of writing; they are re-read against the final release tree before the tag.

🤖 Generated with Claude Code

How much memory a run can use: the controls (environment variables, run_json.py
flags), the fixed resident bounds, the scratch formula with a worked example,
and what grows with bars, trades and open orders. Linked from the docs index
next to "Scratch tables". Docs only; no code, no behaviour change.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@luisleo526
luisleo526 merged commit 6d2bf95 into main Oct 26, 2026
12 checks passed
luisleo526 added a commit that referenced this pull request Oct 27, 2026
…it (resident, disk, jailed arena) (#556)

memory: state the report-memory bound exactly and add the proofs for it (resident, disk, jailed arena)

The memory-management page said the default sink route keeps the equity
curve out of memory, but not what still grows with the bar count, and that
a jailed run without --report-sink-fd has no sink. It now states, with
sources: the resident equity-report buffers are bounded on the default
unjailed sink route; the curve scratch costs 24 B per script bar on disk
(6 MiB at 262,144 and 48 MiB at 2,097,152 bars) with the E output and the
JSON artifact; in a jailed run the scratch is the RAM-backed arena, so the
same 24 B per script bar are memory there, sized by the hosting wrapper and
failing closed with arena_exhausted; sink-off callers keep the native curve
plus an owning report copy. "O(1) in bars" is not claimed for the jailed
arena or the sink-off path.

Removed text, named. The page text below was introduced by 6d2bf95 (docs:
add the memory-management page, #531) and 30e9f9a (docs: memory-management
page re-read against main after the 1.8.0 changes, #546):
- the section "Use the report sink for bounded memory" is replaced by
  "Report memory: resident buffers, disk and the jailed arena" (its line
  citations were re-read and corrected);
- the phrase "a bounded default route is planned, with no release date" and
  the sentence "Install a report sink; no release date for the default
  route" are removed from the report-memory table (the page documents the
  route as it is);
- the two sink rows of that table are reworded to separate resident buffers
  from disk and arena growth; the sentences about the heap bound of the
  equity curve and the ledger, the sink file growth, and the legacy FULL sink
  route are rewritten into the new section, and the release container's
  default profile sentence of 30e9f9a is replaced by the route table.

Tests: a two-size resident proof for the report scratch curve (N = 262,144
and 8N script bars, RSS growth within 1 MiB, scratch growth exactly 24 B per
added bar, metric block bit-equal to the sink-off result); a corpus
projection test that the temporary-sink route and the sink-off route give
bit-identical metrics and diagnostics on real modules (it replaces the
named-sink summary comparison on corpus modules that this test did before,
and still skips when its manifest variable is unset). Both report-memory
rows are pinned as slow rows (61 pinned names) and registered before the
CR-150 include block. The binary-route peak-RSS row is left out of this
change: its 9-minute wall exceeds the CI leg budget.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

ci: count the array-growth slow row with the report-memory slow row (62 pinned names)

Both the report-memory change and the paged array store added one pinned slow
row each on top of 60, and git merged the two identical edits of the count
into 61. The pinned set now holds 62 names: 43 from the PINEFORGE_PR_SLOW_TESTS
loop and 19 direct CMake LABELS assignments.

The test assertion on len(PINNED_SLOW_TEST_ROWS) and the four sentences in
docs/ci.md that state the count and its split (introduced by 3ea5a0d, ci:
split the long test rows, shard the full CI legs and run them on the larger
hosted runners, #544, and rewritten by the report-memory change) are updated
to 62 and 43 + 19. No other text is removed.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

ci: update the slow-row count literals in test_ci_preflight to 62, 43 and 19

test_documented_slow_row_counts_match_literal_population pins the three
numbers that docs/ci.md states for the pinned slow rows (all, from the
PINEFORGE_PR_SLOW_TESTS loop, direct LABELS assignments) and two sentences that
repeat them. The previous commit moved the documented count from 61 to 62
(43 + 19) but left this test at 61, 43 and 18, so test_ci_preflight_slice_6
failed on every profile. The test now states 62, 43 and 19. Only those literals
change.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

memory: move three docker/run_json.py citations after the bulk equity copy

The bulk path-sink equity copy added code above the report-sink helpers in
docker/run_json.py, so the three line ranges the memory-management page cites
for _ReportSink, _build_sink_report and _prepare_stream_report no longer
contained their symbols (doc-anchors: SYMMISS). The ranges now read 6183-6220,
7102-7148 and 6983-7003. Citation numbers only: no sentence is added, removed
or reworded.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>


Also rewritten: the 60-name slow-row counts in docs/ci.md introduced by 3ea5a0d (ci: split the long test rows ..., #544), now 62 = 43 + 19 (the report-memory row and #547's array-growth row); the Pine product host row of the report-memory table in docs/pages/memory-management.md introduced by 27cd77b (engine 1.8.0: cohort tables ..., #541), reworded to separate resident buffers from disk and arena growth.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant