Skip to content

report: opt-in replay extension (raw realized P&L, settlement markers, equity members, companion stream) - #607

Closed
luisleo526 wants to merge 16 commits into
mainfrom
report-replay-companion
Closed

luisleo526 wants to merge 16 commits into
mainfrom
report-replay-companion

Conversation

@luisleo526

Copy link
Copy Markdown
Collaborator

Adds, strictly opt-in (--report-replay, --report-replay-sink), what a validator needs to replay a run exactly:

  • the raw whole-lot realized P&L, with the initial capital and the point value, through one additive non-virtual accessor (the ABI version is unchanged);
  • for each closed trade, the original settlement index and a recut marker, kept through the global-exit recut, the same-bar sort and sealing;
  • for each equity point, the replay members;
  • a separate companion stream (pf-report-replay/v1) with its own count, hash and trailer, bound to the primary sink digest.

With the extension off, the primary sink bytes and the default JSON are unchanged; a baseline byte-identity row checks this. Tests: real-engine runs, vectors, companion grammar and negatives. The documentation changes the revert guard reports are anchor line-number updates only, and the commit message names them.

🤖 Generated with Claude Code

…, equity members, companion stream)

Adds, strictly opt-in (--report-replay, --report-replay-sink), what a
validator needs to replay a run exactly:

- the raw whole-lot realized P&L (net_profit_sum_), with the initial
  capital and the point value, through one additive nonvirtual accessor
  (strategy_net_profit_sum; ABI version unchanged);
- per closed trade the original physical-close settlement index and a recut
  marker (parent id as a decimal string, slice index and count, recomputed
  fields, child index, transaction and whole-lot operands, fee operands),
  retained through the global-exit recut, the same-bar sort and sealing;
- per equity point the settlement count, the raw sum, the equity basis and
  the term before capital;
- a separate companion stream (pf-report-replay/v1) with its own count, hash
  and trailer, bound to the primary sink digest; the primary sink bytes and
  the default JSON are unchanged when the extension is off;
- run_json and pf_sink pass-through, and the companion reader/validator that
  checks the exact replay in bits (left fold from +0.0 of the whole-lot P&L
  per settlement index, no tolerance);
- tests (real-engine runs, vectors, companion grammar and negatives, a
  baseline byte-identity row), the documentation, and the registrations the
  repository guards require for the new surface.

Documentation line anchors were re-pointed to the moved lines. The sentence
changes the revert guard reports in docs/design/native-feature-parity.md
(introduced by 1dd5430 and 91353de) and docs/native-refactor-progress.md
(introduced by d093560) are line-number updates of existing anchors, plus the removal of a
level label from one table cell; no documented fact is changed.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Comment thread docker/run_json.py
if not self.named:
try:
os.unlink(self.path)
except FileNotFoundError:
luisleo526 and others added 15 commits October 28, 2026 15:57
…hanged by the replay extension

Review of 052deae found two defects in the replay extension, both fixed here:

- The extension reordered ReportSinkState and inserted an owning pointer
  into it, which moved every BacktestEngine member after report_sink_state_
  without an epoch change. The original member layout is restored exactly;
  the replay state now lives in a library-owned registry keyed by the sink
  state's address (copies clone configuration, moves rekey, destruction
  erases), and the inline recording code is back to the original tokens.
  A layout test pins the size and every offset of ReportSinkState and the
  position of the next member.
- A public equity mode STREAMED requested without a replay reached the
  replay recorder and dereferenced a null owner, and the mode getter no
  longer returned what the setter accepted. STREAMED without a replay now
  takes the ordinary full-capture path (same bytes as FULL), every replay
  hook tests for an actual owner, and a row covers it.

Also added: a row that a failed run leaves a companion without a trailer,
a row for the equity term with a nonzero range-end open lot, comments for
the sink reconfiguration move and the companion open flags, and the
registrations the repository guards need for the moved containers. The
documentation changes only update the sentences and line anchors introduced
by 052deae.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
native_execution_consumer.cpp reached ReportReplay members through the
forward declaration in engine.hpp; the two equity-recording call sites now
use the existing wrapper methods (same lines, no line shifts), which are
defined where the type is complete.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Adds a ctest row that runs run_json WITHOUT any replay switch on three fixed
scripts (a FIFO re-cut group, an ANY re-cut group, an ordinary close with a
report-only terminal row) and compares the default JSON report and the
pf-report-sink/v1 stream, as raw bytes, with digests recorded from the
engine before the extension (docker/replay_off_golden.json). The row fails if
any default-path byte changes and prints the digests it measured.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…-price vector

The default JSON and primary sink stream of the three fixed scripts now carry
the sha256 and byte count measured on main 95f4fd1 (record run rj-20261028t103035-87ee0b,
six FAIL lines against the empty golden, digests taken from them). The row
test_replay_off_golden is red if any default-path byte changes.

New fixture shape nondyadic: three separate unamended closes at decimal
tenths (100.7, 101.7, 102.9 entries) with a 0.07 per-contract commission and
one held lot. The oracle follows the engine operation order in plain binary64.
The test compares bits of each presented pnl, the left fold from +0.0 against
the replay net profit sum and every Q4, and shows that a 10-significant-digit
rounded fold differs, so the vector can tell rounded operands from exact ones.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…a mismatch

The first run on bbc3cb5 matched the three primary sink streams byte for
byte but not the three default JSON reports: same sizes, different digests.
The report names the build that produced it (engine.version_string, a git
describe, and engine.commit_sha), so a build of another commit can never share
those bytes. The check now replaces those two strings by a fixed token before
hashing; every other byte is compared as produced. On a mismatch the raw text
is printed so it can be diffed against the record. The default JSON digests
are re-recorded from main with the same masking.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The previous mask looked for engine.version_string and engine.commit_sha in
the report. On main the run failed with "default report has no engine identity
block": the report holds the build identity only inside fingerprint.token
(base64 of the canonical provenance JSON) and fingerprint.digest (sha256 of
those bytes). Both are now replaced by a fixed token in the raw bytes, and the
decoded provenance with only the engine version_string and commit_sha blanked
is appended in canonical form, so the rest of the fingerprint is still
compared. Every other byte is compared as produced.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…he build identity masked

The default JSON digests are recomputed on main 95f4fd1 with the corrected
mask (fingerprint token and digest replaced, decoded provenance appended with
only the engine version_string and commit_sha blanked). The three primary
sink digests are unchanged and match the first head run byte for byte.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… as well

The head run differed from main in the default JSON only by the short commit
in the report's plain fingerprint.provenance.engine block (commit_sha
95f4fd1 on main, the head's own short commit on the head); version_string and
everything else were equal, sizes equal. The check now also replaces the plain
engine version_string and commit_sha members (each must occur exactly once)
by a fixed token, besides the token and digest. All other bytes are compared
as produced.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…y build-identity member masked

Recorded on main 95f4fd1 with the check that masks the fingerprint token,
digest and the plain engine version_string and commit_sha members. The three
primary sink digests are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…off-byte pin

The mask replaced the fingerprint token and digest by a fixed token and
compared only the decoded provenance, so a change of the token's spelling
(key order, separators, number spelling) or of the digest scheme could pass.
The check now also verifies that the digest is sha256 of the token bytes,
replaces only the two engine strings inside the token bytes, and compares the
sha256 of the resulting bytes and the digest length. The decimal-price test
comment now says what the assertion proves (at least one rounded P&L differs
and the folded total differs).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…token bytes compared

Recorded with the check that also compares the fingerprint token bytes and
digest scheme. The same six digests were measured on main 95f4fd1 and on
main 1654497 (two runs), so the default report and the sink stream did not
move across those commits. The primary sink digests are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Brings the branch onto the combined 1.8.0 batch. Conflicts resolved:

- src/engine_report.cpp, src/source/pine_strategy_host.cpp: the replay
  recording hooks, tagged pending mode and recut annotations are kept inside
  main's current streaming-metrics, retirement-capability and exit-group
  cache structure. The default path is unchanged.
- ci/cr150-source-bounds.yaml, ci/cr150-source-sites.json and the source
  reading files: main's rows and the replay branch's 18 groups and 43 sites
  are both kept; the replay identifiers are renumbered above main's and every
  moved source coordinate is re-read from the merged files.
- Twelve documentation files: main's plain wording is kept; the replay
  additions are restated in it and every citation points at the merged line.
  The ABI is main's version 5, unchanged by the extension.
- scripts/test_cr150_source_gate.py: the open-group count is 1483 groups of
  main plus the 18 groups of the replay branch, 1501 (the branch's own earlier
  commit changed the same pin from 1465 to 1483 for the same reason).
- tests/test_replay_abi.cpp pins ABI 5; the real-engine fixture states the
  module capability main's retirement fixtures state.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Main moved by one commit (the pin that the repository's own workflow files
pass its admission linter) after the previous merge. It changes only
docs/ci.md, scripts/ci_preflight.py and its test, so the merge is clean and
no citation of the replay extension moves.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ified sink count

Main's fused sink path returns the trades and equity arrays as spooled
streams without a resident length. The replay join and the empty-array
normalisation asked len() of them and failed with "a streamed array has no
resident length" on the sink route. The length is now taken from the
verified sink count (T for trades, E for equity points) when the array has
no resident length; resident arrays keep len(). The default path is unchanged.

Also moves two source-line citations of the feature-parity page by one line
(exit_comment 1110, exit_from_bracket 1635) as the anchor check reports.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… returns encoded columns

After the length fix the sink route still joined zero rows: main's fused
path returns the trades and equity arrays as already encoded JSON columns
that hold no rows to iterate, so the replay join found its R records exceeding
the JSON trades. When replay rows are requested, the arrays are now built
from the verified sink records (trades through the existing trade decoder,
equity points as dicts), exactly as the counted arrays of the earlier path.
The default path and the named-sink summary are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@luisleo526

Copy link
Copy Markdown
Collaborator Author

Folded into #632, the combined 1.8.0 batch-2 pull request. The branch is kept.

@luisleo526 luisleo526 closed this Oct 29, 2026
luisleo526 added a commit that referenced this pull request Oct 29, 2026
…ion, metrics-only mode (#632)

Combined second batch for 1.8.0. Each member was reviewed and proven on its own branch; this branch merges them onto main and is verified as one tree.

## Members (merged in this order)

- Keep a reversal's newly credited opening when its entry id had opposite-side history (fixes wrong positions, uncapped pyramiding and missing closed-trade rows after a reversal on a reused entry id; affected releases v1.1.0 through v1.7.0).
- Window buffers for technical-analysis series kept on main.
- Report channel contract and hosting jail budget contract (docs; #613, #614).
- Annualization diagnostic uses the engine's initial capital on every report path.
- Opt-in report replay companion (#607).
- Metrics-only run mode (#630).
- Comment-only rewording of shipped source comments.
- Source clones skip the in-repo `.ccache` directory, so a whole-tree copy no longer races a running compile.
- In-process iteration of the default-path report columns: `report["trades"]` and `report["equity_curve"]` iterate lazily from the report spool again, report their length and support integer indexing; iteration must happen before the report's resources close.
- CHANGELOG entries for the above, and re-pointed source-line citations in two documentation pages (line numbers only).
- The report-sink trade-identity test reference is re-recorded: the reversal fix changes the per-bar state hash input (open-position bookkeeping), while no trade or equity row changes; the CHANGELOG says so.

## Not in this batch

The live runner's report checkpoints (#598) and report accumulator (#617) move to the next minor release: testing showed a timing case that can leave a delivered order unacknowledged and a crash for modules whose handle is not an engine. Their pull requests stay open as drafts.

## Verification

- Native live suite (all native shards, thread and address sanitizer legs) on this tree's code.
- Full verify and the corpus parity sweep on this exact head.
- Every member was reviewed independently before it was merged here.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant