Repository navigation
report: opt-in replay extension (raw realized P&L, settlement markers, equity members, companion stream) - #607
Closed
luisleo526 wants to merge 16 commits into
Closed
luisleo526 wants to merge 16 commits into
luisleo526 wants to merge 16 commits into
Conversation
…, equity members, companion stream) Adds, strictly opt-in (--report-replay, --report-replay-sink), what a validator needs to replay a run exactly: - the raw whole-lot realized P&L (net_profit_sum_), with the initial capital and the point value, through one additive nonvirtual accessor (strategy_net_profit_sum; ABI version unchanged); - per closed trade the original physical-close settlement index and a recut marker (parent id as a decimal string, slice index and count, recomputed fields, child index, transaction and whole-lot operands, fee operands), retained through the global-exit recut, the same-bar sort and sealing; - per equity point the settlement count, the raw sum, the equity basis and the term before capital; - a separate companion stream (pf-report-replay/v1) with its own count, hash and trailer, bound to the primary sink digest; the primary sink bytes and the default JSON are unchanged when the extension is off; - run_json and pf_sink pass-through, and the companion reader/validator that checks the exact replay in bits (left fold from +0.0 of the whole-lot P&L per settlement index, no tolerance); - tests (real-engine runs, vectors, companion grammar and negatives, a baseline byte-identity row), the documentation, and the registrations the repository guards require for the new surface. Documentation line anchors were re-pointed to the moved lines. The sentence changes the revert guard reports in docs/design/native-feature-parity.md (introduced by 1dd5430 and 91353de) and docs/native-refactor-progress.md (introduced by d093560) are line-number updates of existing anchors, plus the removal of a level label from one table cell; no documented fact is changed. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
| if not self.named: | ||
| try: | ||
| os.unlink(self.path) | ||
| except FileNotFoundError: |
…hanged by the replay extension Review of 052deae found two defects in the replay extension, both fixed here: - The extension reordered ReportSinkState and inserted an owning pointer into it, which moved every BacktestEngine member after report_sink_state_ without an epoch change. The original member layout is restored exactly; the replay state now lives in a library-owned registry keyed by the sink state's address (copies clone configuration, moves rekey, destruction erases), and the inline recording code is back to the original tokens. A layout test pins the size and every offset of ReportSinkState and the position of the next member. - A public equity mode STREAMED requested without a replay reached the replay recorder and dereferenced a null owner, and the mode getter no longer returned what the setter accepted. STREAMED without a replay now takes the ordinary full-capture path (same bytes as FULL), every replay hook tests for an actual owner, and a row covers it. Also added: a row that a failed run leaves a companion without a trailer, a row for the equity term with a nonzero range-end open lot, comments for the sink reconfiguration move and the companion open flags, and the registrations the repository guards need for the moved containers. The documentation changes only update the sentences and line anchors introduced by 052deae. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
native_execution_consumer.cpp reached ReportReplay members through the forward declaration in engine.hpp; the two equity-recording call sites now use the existing wrapper methods (same lines, no line shifts), which are defined where the type is complete. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Adds a ctest row that runs run_json WITHOUT any replay switch on three fixed scripts (a FIFO re-cut group, an ANY re-cut group, an ordinary close with a report-only terminal row) and compares the default JSON report and the pf-report-sink/v1 stream, as raw bytes, with digests recorded from the engine before the extension (docker/replay_off_golden.json). The row fails if any default-path byte changes and prints the digests it measured. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…-price vector The default JSON and primary sink stream of the three fixed scripts now carry the sha256 and byte count measured on main 95f4fd1 (record run rj-20261028t103035-87ee0b, six FAIL lines against the empty golden, digests taken from them). The row test_replay_off_golden is red if any default-path byte changes. New fixture shape nondyadic: three separate unamended closes at decimal tenths (100.7, 101.7, 102.9 entries) with a 0.07 per-contract commission and one held lot. The oracle follows the engine operation order in plain binary64. The test compares bits of each presented pnl, the left fold from +0.0 against the replay net profit sum and every Q4, and shows that a 10-significant-digit rounded fold differs, so the vector can tell rounded operands from exact ones. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…a mismatch The first run on bbc3cb5 matched the three primary sink streams byte for byte but not the three default JSON reports: same sizes, different digests. The report names the build that produced it (engine.version_string, a git describe, and engine.commit_sha), so a build of another commit can never share those bytes. The check now replaces those two strings by a fixed token before hashing; every other byte is compared as produced. On a mismatch the raw text is printed so it can be diffed against the record. The default JSON digests are re-recorded from main with the same masking. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The previous mask looked for engine.version_string and engine.commit_sha in the report. On main the run failed with "default report has no engine identity block": the report holds the build identity only inside fingerprint.token (base64 of the canonical provenance JSON) and fingerprint.digest (sha256 of those bytes). Both are now replaced by a fixed token in the raw bytes, and the decoded provenance with only the engine version_string and commit_sha blanked is appended in canonical form, so the rest of the fingerprint is still compared. Every other byte is compared as produced. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…he build identity masked The default JSON digests are recomputed on main 95f4fd1 with the corrected mask (fingerprint token and digest replaced, decoded provenance appended with only the engine version_string and commit_sha blanked). The three primary sink digests are unchanged and match the first head run byte for byte. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… as well The head run differed from main in the default JSON only by the short commit in the report's plain fingerprint.provenance.engine block (commit_sha 95f4fd1 on main, the head's own short commit on the head); version_string and everything else were equal, sizes equal. The check now also replaces the plain engine version_string and commit_sha members (each must occur exactly once) by a fixed token, besides the token and digest. All other bytes are compared as produced. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…y build-identity member masked Recorded on main 95f4fd1 with the check that masks the fingerprint token, digest and the plain engine version_string and commit_sha members. The three primary sink digests are unchanged. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…off-byte pin The mask replaced the fingerprint token and digest by a fixed token and compared only the decoded provenance, so a change of the token's spelling (key order, separators, number spelling) or of the digest scheme could pass. The check now also verifies that the digest is sha256 of the token bytes, replaces only the two engine strings inside the token bytes, and compares the sha256 of the resulting bytes and the digest length. The decimal-price test comment now says what the assertion proves (at least one rounded P&L differs and the folded total differs). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…token bytes compared Recorded with the check that also compares the fingerprint token bytes and digest scheme. The same six digests were measured on main 95f4fd1 and on main 1654497 (two runs), so the default report and the sink stream did not move across those commits. The primary sink digests are unchanged. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Brings the branch onto the combined 1.8.0 batch. Conflicts resolved: - src/engine_report.cpp, src/source/pine_strategy_host.cpp: the replay recording hooks, tagged pending mode and recut annotations are kept inside main's current streaming-metrics, retirement-capability and exit-group cache structure. The default path is unchanged. - ci/cr150-source-bounds.yaml, ci/cr150-source-sites.json and the source reading files: main's rows and the replay branch's 18 groups and 43 sites are both kept; the replay identifiers are renumbered above main's and every moved source coordinate is re-read from the merged files. - Twelve documentation files: main's plain wording is kept; the replay additions are restated in it and every citation points at the merged line. The ABI is main's version 5, unchanged by the extension. - scripts/test_cr150_source_gate.py: the open-group count is 1483 groups of main plus the 18 groups of the replay branch, 1501 (the branch's own earlier commit changed the same pin from 1465 to 1483 for the same reason). - tests/test_replay_abi.cpp pins ABI 5; the real-engine fixture states the module capability main's retirement fixtures state. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Main moved by one commit (the pin that the repository's own workflow files pass its admission linter) after the previous merge. It changes only docs/ci.md, scripts/ci_preflight.py and its test, so the merge is clean and no citation of the replay extension moves. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ified sink count Main's fused sink path returns the trades and equity arrays as spooled streams without a resident length. The replay join and the empty-array normalisation asked len() of them and failed with "a streamed array has no resident length" on the sink route. The length is now taken from the verified sink count (T for trades, E for equity points) when the array has no resident length; resident arrays keep len(). The default path is unchanged. Also moves two source-line citations of the feature-parity page by one line (exit_comment 1110, exit_from_bracket 1635) as the anchor check reports. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… returns encoded columns After the length fix the sink route still joined zero rows: main's fused path returns the trades and equity arrays as already encoded JSON columns that hold no rows to iterate, so the replay join found its R records exceeding the JSON trades. When replay rows are requested, the arrays are now built from the verified sink records (trades through the existing trade decoder, equity points as dicts), exactly as the counted arrays of the earlier path. The default path and the named-sink summary are unchanged. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Collaborator
Author
|
Folded into #632, the combined 1.8.0 batch-2 pull request. The branch is kept. |
luisleo526
added a commit
that referenced
this pull request
Oct 29, 2026
…ion, metrics-only mode (#632) Combined second batch for 1.8.0. Each member was reviewed and proven on its own branch; this branch merges them onto main and is verified as one tree. ## Members (merged in this order) - Keep a reversal's newly credited opening when its entry id had opposite-side history (fixes wrong positions, uncapped pyramiding and missing closed-trade rows after a reversal on a reused entry id; affected releases v1.1.0 through v1.7.0). - Window buffers for technical-analysis series kept on main. - Report channel contract and hosting jail budget contract (docs; #613, #614). - Annualization diagnostic uses the engine's initial capital on every report path. - Opt-in report replay companion (#607). - Metrics-only run mode (#630). - Comment-only rewording of shipped source comments. - Source clones skip the in-repo `.ccache` directory, so a whole-tree copy no longer races a running compile. - In-process iteration of the default-path report columns: `report["trades"]` and `report["equity_curve"]` iterate lazily from the report spool again, report their length and support integer indexing; iteration must happen before the report's resources close. - CHANGELOG entries for the above, and re-pointed source-line citations in two documentation pages (line numbers only). - The report-sink trade-identity test reference is re-recorded: the reversal fix changes the per-bar state hash input (open-position bookkeeping), while no trade or equity row changes; the CHANGELOG says so. ## Not in this batch The live runner's report checkpoints (#598) and report accumulator (#617) move to the next minor release: testing showed a timing case that can leave a delivered order unacknowledged and a crash for modules whose handle is not an engine. Their pull requests stay open as drafts. ## Verification - Native live suite (all native shards, thread and address sanitizer legs) on this tree's code. - Full verify and the corpus parity sweep on this exact head. - Every member was reviewed independently before it was merged here. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds, strictly opt-in (
--report-replay,--report-replay-sink), what a validator needs to replay a run exactly:pf-report-replay/v1) with its own count, hash and trailer, bound to the primary sink digest.With the extension off, the primary sink bytes and the default JSON are unchanged; a baseline byte-identity row checks this. Tests: real-engine runs, vectors, companion grammar and negatives. The documentation changes the revert guard reports are anchor line-number updates only, and the commit message names them.
🤖 Generated with Claude Code