Repository navigation
live runner: a delta report stream by default, and report-path work that does not rescan closed trades - #594
Closed
luisleo526 wants to merge 4 commits into
Closed
luisleo526 wants to merge 4 commits into
luisleo526 wants to merge 4 commits into
Conversation
…hat does not rescan closed trades After every input the runner scanned the whole closed-trade prefix again and, with --report-jsonl, printed the entire cumulative report: quadratic CPU and output over a long run, plus a retained copy of the whole document. - ReportDeltas carries a closed-trade high-water mark, so extraction visits only the trades appended since the last update. The retained cumulative document exists only in full mode; the default keeps scalar state and array counts. - --report-jsonl now emits one pineforge-native-report-delta/v1 document per committed input, the same payload the ledger stores. The cumulative document is --report-jsonl=full. This changes the stream shape for consumers that read the cumulative document; in-repo consumers and the documentation are updated. - The native fill_report call still rebuilds the engine's trade and equity/metrics report for each input. Removing that term needs an engine-side carried accumulator and its own proof that batch reports stay byte for byte identical; the changelog says so. - tests/test_live_report.cpp compares the final cumulative report, applying the delta stream with the test's own applier, with the full-mode document; counts the closed trades visited over runs of N and 10N inputs and the bytes written in delta mode; and checks the option values. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
test_live_report uses the run-failure machinery but its target did not link the library that defines it, so the kernel profile stopped at the link step. It now links the same library as the neighbouring runner tests. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ke the delta stream opt-in An external consumer reads the first line of the runner's stdout and refuses the engine unless it is the cumulative pineforge-native-report/v1 document, so the default of --report-jsonl cannot change shape in this release. - Plain --report-jsonl and --report-jsonl=full print the cumulative document per committed input, byte for byte as before; --report-jsonl=delta prints the pineforge-native-report-delta/v1 document. Any other value is refused with the usage error. The documentation states each mode's per-input bound: the delta stream is O(new rows), the cumulative stream is O(cumulative document). - The closed-trade high-water mark stays in every mode, so the runner no longer rescans the closed-trade prefix per input; the retained cumulative document exists only where the cumulative stream needs it. - New tests: the delta stream's first line carries an integer input_cursor of at least 1 at the same commit point as the cumulative stream's first line; a run without --report-jsonl prints exactly one stdout line, the exit summary; a revised last closed trade, the closed-trade prefix regression error and the option values are covered. This rewrites passages introduced by 4342247: the sentences that called plain --report-jsonl the delta stream ("Delta JSONL is the default", "request --report-jsonl=full to retain that cumulative output") and the heading "Per-input report-path work is flat in the appended rows" now say the default is cumulative, delta is opt-in, and that the runner no longer rescans the closed-trade prefix per input while the delta output is linear in the appended rows. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The delta stream line carries input_cursor as its first top-level key and the stored ledger row has no such key; the runner README and the changelog said only "an additional top-level integer". They now state the key order and that the stored row differs. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Oct 28, 2026
…aling, UDT graphs, metrics replay and the CI admission fix (#616) This lands twelve engine pull requests as one change, verified and measured once on the combined tree: - #593 scratch: a fixed write-through read cache for the adapter's id tables - #594 live runner: a delta report stream by default, and report-path work that does not rescan closed trades - #597 live runner: typed pre-growth stop for the ledger budget and flat pending counts - #608 run_json: reserve the report channel, so strategy output on fd 1 cannot corrupt a report - #586 Pine UDT object graphs on the paged object Store - #578 native order: dormant working book, flat per-point cost for never-filled orders - #579 adapter: skip roster passes and empty-family scans that cannot change anything - #599 adapter: read projected pending orders through one cached duplicate index - #563 adapter: gate the reversal exit cancel on candidate owners instead of every historical id - #591 report sink: retire sealed closed-trade rows on sink routes when the module opts in - #596 metrics: stream the bar Sharpe and Sortino through an ordered, packed return replay - #615 ci: accept release.yml's matrix runner under its fetched path in PR admission Fixes found while testing the combined tree are in the final commits: the retiring ledger's cache sizing, a residency witness that bounds sink-run heap by the declared cache cap and checks zero growth at 100k, 200k and 1M trades, a test fix for a descriptor reuse race, and a standing-orders timing grid whose 20,000-order point is tracked in #621. The member pull requests are closed as landed here.
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft. The live runner no longer rescans the whole closed-trade prefix after every input, and its JSONL report stream becomes a delta stream by default.
ReportDeltaskeeps a closed-trade high-water mark, so each update visits only the trades appended since the last one. The retained cumulative document exists only in full mode.--report-jsonlnow emits onepineforge-native-report-delta/v1document per committed input, the same payload the ledger stores. The previous cumulative document is--report-jsonl=full.fill_reportcall still rebuilds the trade and equity/metrics report for each input. Removing that needs an engine-side accumulator with its own byte-identity proof; the changelog says so.tests/test_live_report.cppchecks that applying the delta stream reproduces the full-mode document. It also counts the closed trades visited over N and 10N inputs.🤖 Generated with Claude Code