Trading metrics + equity curve in backtest report (ABI v2) - #31

ABI v2 (#31) added pf_report_t::metrics (all/longs/shorts trade stats +
the equity-curve-derived block) and the per-script-bar equity_curve, and
run_json.py already mirrored both structs via ctypes — but
build_report_dict never serialized them, so every JSON consumer (the MCP
backtest tools) only ever saw `summary` + `trades`. Wire both blocks into
the report, plus the ABI v2 per-trade fields (commission,
entry_bar_index, exit_bar_index).

Non-finite metrics (the engine's empty/zero-denominator NaN convention)
collapse to null via _num: json.dump would otherwise emit a bare `NaN`
token that a strict downstream JSON.parse rejects.

Verified end-to-end against the 0.10.2 image with the patched shim
mounted: report is strict JSON; metrics.all/longs/shorts/equity and a
per-bar equity_curve are present; empty-side stats serialize as null.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

…to calling-bar boundary

request.security(sym, tf, ta.<fn>(...)[k], lookahead=barmerge.lookahead_on)
where tf is FINER than the script/chart TF (e.g. so2TF="5" read from a 15m
chart) fed the read-before-push history-offset machinery (added for #30/#31)
one native-security-period value late instead of one CALLING-bar late: the
security's own aggregator completes R = script_seconds/requested_seconds
times per calling bar, and hist.push() fired (unconditionally on
is_complete) on every one of those completions, so hist[0] silently drifted
to "one security-period behind the LAST completion of THIS calling bar"
instead of "the last completion of the PREVIOUS calling bar".

Add SecurityEvalState::publish_gate_tf_seconds (validate_security_timeframes):
nonzero only for a plain (non-security_lower_tf) request.security whose
target TF is strictly finer than script_tf and evenly divides it. In
feed_security_eval_state's aggregator branch, gate the is_complete flag
passed to evaluate_security() on wall-clock bucket-end alignment with the
script_tf boundary, instead of passing the raw completion flag through
unconditionally. The underlying TA state (compute/recompute dispatch) is
driven by current_sub_bar_count, independent of this flag, so intermediate
completions still advance the indicator correctly -- only the exposed
history buffer's publish cadence changes. No-op (field stays 0) for every
coarser-or-equal-TF security and every request.security_lower_tf call.

3commas-3commas-triple-rsi-dca-long-risk-adjusted-averaging-strategy
(so2Rsi = request.security(sym, "5", ta.rsi(close,7)[1], lookahead=
barmerge.lookahead_on) on a 15m chart): 50.5% -> 100.0% match vs
TradingView (108/108 trades, 100% exact price). Full corpus
(252 strategies) + ctest (77) unchanged: excellent=251/anomaly=1,
tier-for-tier identical to baseline. Spot-checked every other
lookahead_on + history-offset request.security call site in the standard
corpus (6 other slugs, all coarser-than-chart target TFs, e.g. "D"/"W"/
240/60) -- byte-identical engine_trades.csv/engine_verify.json output,
confirming zero blast radius outside this one finer-than-chart case.

…-release 3e39d64 (#343)

Re-render the marked README values against the hub's canonical facts at 3e39d64
(the squash of release hub #31): the active scoreboard is
pineforge-parity-baseline-20261005-input-reanchor-781b8f00 (engine 3c41452,
codegen 39545379), 7,975 excellent / 14 strong of 7,989 graded, none below
strong. Release scoreboards keep their own numbers (release 1.2.0: 7,970
excellent / 19 strong). Only values inside pf markers change.

Co-authored-by: Claude Sonnet 5.5 <[email protected]>

#456)

* report: fail with report_result_invalid when a report metric overflows; annualise cagr proportionally under one day, with a named form flag (CR-202, owner W-73)

Squash of the CR-202 work on engine main 0212cab. Its single parent is engine main, as the GitHub squash writes it.

What changes
- A report metric whose computed value overflows (buy_hold_return, every other report metric) fails the run with the typed failure report_result_invalid, reason nonfinite, instead of writing null.
- Proportional annualisation (owner ruling W-73; docs/design/selected-window-wire-v1.3-revision-3.1.md, docs/pages/report-schema.md). span_years = (last bar time - first bar time) / (365.25 * 86400 * 1000 ms), as before. Under one day (span_ms < 86 400 000) cagr = (E/E0 - 1) * (1/span_years) * 100, the simple proportional form, always, and calmar = cagr / max_equity_drawdown_pct follows. From one day on cagr is the compounded form, unchanged; only if that value is not finite the simple form is used instead (a rare ranking inversion across the one-day boundary, documented). sharpe_bar and sortino_bar use the same formulas on every span and are never absent by span; sharpe_tv and sortino_tv are unchanged. No null for these four on a valid run; every other non-finite value is the typed report_result_invalid.
- The public report names the form under diagnostics.annualization (outside the 135 digest leaves): cagr = "simple_proportional_subday" or "simple_proportional_overflow", and calmar carries the same string when it is not null. The writer derives it from the curve it holds; for a run of one day or more whose compounded form is not finite and whose first mark is not proven to be the initial capital the cagr is still the simple value and only the overflow flag is absent (documented; a runtime export of the capital is carded for a later release). The frozen report structures gain no member.
- C host (src/native_c_host.cpp): a report failure after a completed run is recorded and returns E_STATE, a report failure after an earlier latched failure returns E_RUN_FAILED with that failure; in both the caller's report struct is defined (zeroed local report freed on a throw) instead of half written.
- The native runner keeps its own report format: undefined metrics stay the string "NaN" (runner/README.md); only an infinite value is the typed failure there.
- Disclosed behaviour: an infinite first-open or last-close price and a non-finite selected anchor are now report_result_invalid (feed admission makes the first unreachable).
- kNeutralReportDigest in tests/test_native_anchored_legs.cpp is a lab harvest of the neutral 15 one-minute bar run (its cagr is the simple proportional value on a 14-minute span); the value is taken from the lab log under the fixture authority of CORE-TOP.

Documentation sentences reworded or replaced by this change, each named for scripts/check_doc_reverts.py (the sentence's introducing commit, then its first words):
- 1bd075c (Trading metrics + equity curve in backtest report, #31): "See the per-field doxygen in `<pineforge/pineforge.h>` for the exact rule on every field." in docs/pages/report-schema.md, now "in `<pineforge/pineforge.h>` for the existing field domains; the rule below is authoritative" for cagr on a span under one day and when its compounded form overflows.
- 1b08c61 (EN-133, #384): "net_pnl, avg_trade, best_trade, worst_trade and max_drawdown, must be finite or the run fails with report_result_invalid, field naming the corresponding trades.<leaf> or summary.<leaf> and reason equal to nonfinite." in docs/pages/report-schema.md, extended with win_rate_pct and the equity-curve equity and open_profit leaves.
- 5691220 (ADR-0001 rewritten against HEAD): "compute_equity_stats answers its all-NaN value with no error" in docs/adr/0001-kernel-adapter-boundary.md, now "answers its defined empty-curve metrics with no error".
- 375795a (shard the ownership manifest-controls test, CR-196): "(9, 9, 9 and 8 for shards 1 to 4 of 4: 8, 8, 8 and 7 cases plus the guard)" and "The separate cheap test_shards_partition_every_case_exactly_once guard runs in every shard" in tests/ownership/README.md, now "(10, 10, 10 and 10 for shards 1 to 4 of 4: 9 cases plus the guard in each)" and "Each shard also runs the cheap test_shards_partition_every_case_exactly_once guard".
- file:line citation digits only, wording kept: ADR-0001 (the "inclusive bar metrics" bullet, the "Reporting." bullet, the strategy_configure_native_v1 row, the magnifier_sub_bars_total row), native-feature-parity rows RP2, RP4, S1b and "A zoned chart-day memo", native-refactor-progress (the quiet-bar cost check and the every_gate_is_taken_both_ways citation), native-engine ("MUST mirror" citation of compute_equity_stats).

Documentation sentences the history of this branch introduced and later reworded or removed (none of them is deleted by this squash against main):
- c9ba507 introduced "Report arithmetic that produces a nonfinite value, including overflow, fails with `report_result_invalid` and reason `nonfinite`." and "The engine checks computed values before they can become a metric absence." in docs/pages/report-schema.md, both kept.
- eba8561 (revision 3.1) introduced "The constant policy list is in `docker/report_annualization.def`, shared by the native metric walk and public report writer." in docs/design/selected-window-wire-v1.3-revision-3.1.md; e963e5e (the earlier squash of this work) introduced the revision 3.1 note's null-rule sentences ("The short-span absence applies to cagr, calmar, sharpe_bar and sortino_bar", "Calmar carries the short-span reason with CAGR even when the curve also has zero drawdown", "span_too_short_to_annualize") and the report-schema.md ones; the owner ruling W-73 withdrew that null rule and they are reworded or replaced in this change.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* tests: keep the base assertion CHECK(!std::isnan(e.cagr)) in test_metrics_l4d (twin parity)

scripts/check_twin_parity.py requires every assertion literal of the base test_metrics to survive in its switched-route twin test_metrics_l4d.cpp. The proportional-annualisation rewrite of test_equity_stats_drawdown_walk dropped `CHECK(!std::isnan(e.cagr));`, so the source-guard-twin-parity stage failed and the verifier-tests stage of preflight read `not-started` for every later stage. The literal is back, unchanged and still true (cagr is defined on every valid run); the reference comparison below it stays.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

* tests: re-pin kNeutralReportDigest to the lab harvest of the proportional-annualisation report

The neutral run (15 one-minute bars, a 14-minute span, KernelRecorded) now reports a finite simple proportional cagr (10000 to about 10005, no drawdown) and main's formulas for sharpe_bar and sortino_bar, so nothing is absent and its report digest moves again from the value pinned for the earlier null rule. Value harvested from lab job rj-20261020t035042-1a9c33 (label cr202-v6c-pre-rel, release step, test_native_anchored_legs:262), log line 2525: "neutral report digest 6216030702059937357, pinned 8139101939074791901". Fixture authority for exactly that printed value: CORE-TOP (2026-10-20 12:29). That test was the only failure of that release run (961 of 962 tests passed).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

---------

Co-authored-by: Claude Sonnet 5.5 <[email protected]>

…1ba62b) on core/cr150-o1-integ b4d2011

One commit carrying the tree of o1w/merge0 0a41361 (the full-history record of the merge-7 line), as a child of core/cr150-o1-integ b4d2011, so the doc-reverts gate judges the whole change against its real parent with this message (CORE-TOP ruling 10-21 18:04: no rewrite of pushed history, no checker exception).

Merged lane heads (all carried in the tree): o1w-contract b19d8ae, o1w-pyjson cbc68d2, o1w-tail 27fcad9, o1w-selected 91fbd0e, o1w-seq 4b84f9d, o1w-tradeout bf024f1, o1w-ledger a87e546, o1w-metrics 3e1e70d, o1w-slabfold d41311c, batch-B 583c221, L8d proof 07c6363, L9b proof b01fe68, batch-C e6f8fcb, Q7 abort d220bd7, Q9 converter 3478350. Merge 7 is engine main 21ba62b. Rulings applied: CORE-TOP A (the four unbacked contract declarations withdrawn until the ledger and metrics lanes define them), B (the two _JSONRecordArray factory forms in the selected-window ownership scanner), E and F (closed-row accessor by copy; indexed report-array read), R13 (main's typed finite failures and proportional sub-day annualization win), the Option A failure-class split.

DOC-REVERTS NAMES: every sentence deletion or revert listed by the doc-reverts gate on preflight 6f8e97 is a deliberate replacement made by an accepted ruling, a regeneration, or a later rewrite of the same page, and is named here by the commit that introduced it:

- 03b2406 (ci: release jobs run on pf-ci-release, #461): the docs/ci.md paragraphs "Push this prepared migration only after both CORE-TOP gates", "Until then a job on that label with no runners only waits, and nothing is pushed.", the PF_CI_LABELS / pre-job hook / release.yml / promote-baseline.yml sentences: rewritten by main's own later edit of that page (21ba62b, #464), taken whole by merge 7.
- a8f84b9 (docs(api): C ABI reference site): docs/pages/report-schema.md "pf_report_t is the single output of every backtest.", "This page maps each field to its meaning and units.", "Heap array, ordered by exit time ascending." rewritten by the O(1) report contract pages.
- efb8c7e (o1w-contract round 2): docs/pages/report-schema.md "C/ctypes clients require an explicitly negotiated provider bridge; there is no new runtime C export in this slice." superseded by the selected-window named-sink contract text of b19d8ae.
- 39a718b (cr150 L6b-1): docs/pages/run-failure-codes.md "The engine could not create or write the report sink." and "The run has no completed sink or valid trailer." replaced by the regenerated report_sink_failed description.
- d220bd7 (cr150-q7): run-failure-codes.md "The existing engine_fault/retryable policy is retained pending a ruling:", "PROPOSED cancelled, retryable false;" and "RETURN TO CORE-TOP for the class ruling." removed by the class ruling (run_aborted only, no cancelled class, no retry field).
- d91c96d (CR-150 route (ii) foundation): run-failure-codes.md "The engine could not create, resize, read or write scratch-table storage." replaced by the split wording (auxiliary tables raise scratch_io_failed; report-sink scratch raises report_sink_failed reason scratch_io).
- e6f8fcb (CR-150 batch-C): docs/design/arena-bound.md "`sequence_index_` `include/pineforge/source/pine_adapter.hpp:N`" and the "`include/pineforge/detail/id_pair_store.hpp:N`;" citation: re-pointed or dropped after the merged source moved.
- 5dbe91c (merge batch-B 583c221): docs/design/native-feature-parity.md "**yes** — `fill_report` engine_report.cpp:N, `get_trade` engine.hpp:N, `compute_trade_stats`" and "**no** — `compute_equity_stats` exits early on an empty curve": single-line pins replaced by the backticked symbol, as the anchor checker instructs.
- 4d6cd2b (CR-150 WL-PTAILFIX): docs/design/consumed-partial-exit-fold.md "The previous-header fail-before compilation of the new unit remains required; its expected first diagnostic is the additional `IdPairStore` template a..." rewritten by o1w-slabfold d41311c.
- d093560 (INT26 wave H): docs/pages/native-engine.md "A C++ host reads a closed row whole, at any time, every `Trade` field included." replaced by the closed-row-access-by-copy row (CORE-TOP ruling E: closed rows are paged, the C++ accessor returns a copy).

DOC-REVERTS NAMES, second set: against this commit's real parent b4d2011 the gate reports 21 unnamed sentence deletions or reverts, all introduced by the seven commits below (preflight rj-20261021t100903-91b0cc, stage doc-reverts). Each is a deliberate rewrite on top of the introducing commit, and each introducer is named here:

- 133f571 ("1.0 prep: CI flake fix, docs-only CI skip, benchmark refresh (supersedes #303, #304, #307)"), eleven docs/ci.md sentences: "The compilation lanes start alongside preflight, as soon as the few-second `changes` job has classified the change", "A pull request or a push to `main` whose change is documentation only runs `preflight` and the documentation build", "Each proof job waits for `changes` alone, a few seconds once a standard runner takes it", "`preflight` waits for nothing.", "`changes` and `preflight` wait for nothing and run on every event", "A documentation-only change starts preflight alone there.", "So does the `changes` job, a two-commit checkout and one `git diff`.", "The same stage pins the documentation-only skip", "The advisory `build` aggregate passes a proof job that succeeded", "A proof job skipped for any other reason fails it", "success": rewritten by main's later CI redesign and carried whole by merge 7 of 21ba62b.
- b2a578c ("docs: make every README and doc page true before the release (DOCS-ENGINE) (#300)"), one docs/ci.md sentence on the PR registration floors after lane FIX-E1E2 (773 Debug and sanitizers, 782 native): superseded by the later floor rows.
- 2394ba2 ("ci: remove every time limit that can fail a run for being slow (#373)"), two docs/ci.md sentences: "The table below reports each job's measured duration and the platform maximum that replaced its old limit" and "The numbers are measured durations, not a two-times multiplier.": replaced by the current duration table.
- 7da487a ("ci/tests: remove the CI time limits; a platform-maximum stop is a HANG to re-run (EN-116)"), two docs/ci.md sentences: "Under the owner rule, no required-check stage, row, step, profile or matrix entry may carry a time limit" and "The native workflow retains only its 360-minute job envelopes, with no verification-step deadline.": superseded by the same redesign.
- b148979 ("selected window: declare 1.3-parity, not 1.3, until a completeness proof exists (1.6.1)"), three sentences, README.md and docs/pages/report-schema.md: "`capabilities.selected_window_contract_revisions` is `[\"1.3-parity\"]` when the runtime supports selected windows", "The `--capabilities-json` probe's `capabilities.selected_window_contract_revisions` is `[\"1.3-parity\"]`", "From 1.6.1 (unreleased), the image declares `1.3-parity` instead": reverted to the 1.3 pin by the later capability ruling (AMD6 1.3-parity only), an accepted reversal.
- 1bd075c ("Trading metrics + equity curve in backtest report (ABI v2) (#31)"), one docs/pages/report-schema.md sentence: "See the per-field doxygen in `<pineforge/pineforge.h>` for the exact rule on every field.": replaced by the O(1) report contract's per-field pages.
- d3cca9e ("Docs: correct migration semantics and harden publication guards"), one docs/design/native-feature-parity.md pin line (`sha256:H` c_abi.cpp:N, `sha256:H` check_c_abi_runtime.py:N, ...): replaced by the symbol-only rows, as the anchor checker instructs.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>