Skip to content

Commit c9ba507

Browse files
luisleo526claude
andcommitted
report: fail with report_result_invalid when a report metric overflows (CR-202)
Report metrics that overflowed (buy_hold_return, cagr and every other product, ratio, sum or annualisation in engine_metrics.cpp, engine_report.cpp, runner/report.hpp and docker/run_json.py) were silently serialised as null. Revision 3 Appendix A:82-83 and :93 allow null only for a defined absence, so an overflowed value now raises the typed report_result_invalid (reason nonfinite) at the computation that produced it. - src/report_value_invalid.hpp: one finite-value guard on the existing coded-failure mechanism. - engine_metrics.cpp / engine_report.cpp / runner/report.hpp / run_json.py: checked reductions and ratios; defined-domain nulls unchanged. - tests/ownership/probes/report/buy_hold_return_overflow: the c18_3 probe, pinned in the manifest and its checker. - tests/test_metrics_overflow.cpp: native boundary and absence controls for all 87 floating metrics; Python serializer expectations updated. Not yet run on the lab: the red-first proof and the gate follow. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
1 parent 375795a commit c9ba507

24 files changed

Lines changed: 1108 additions & 116 deletions

‎docker/run_json.py‎

Lines changed: 45 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@
5454
"longs": { ...pf_trade_stats_t... }, # long trades only
5555
"shorts": { ...pf_trade_stats_t... }, # short trades only
5656
"equity": { ...pf_equity_stats_t... } # sharpe/sortino/cagr/calmar/...
57-
}, # any NaN statistic -> null (see _num)
57+
}, # defined metric absence -> null (see _num)
5858
"equity_curve": [ # ABI v2: one point per script bar
5959
{ "time_ms": int, "equity": float, "open_profit": float },
6060
...
@@ -129,7 +129,8 @@
129129
130130
NaN convention: any metric with an empty/zero denominator is null (JSON has no
131131
NaN); a real computed 0 stays 0. See the report-schema + metrics reference docs
132-
for the per-field meaning of every metrics.* key.
132+
for the per-field meaning of every metrics.* key. Overflow and other computed
133+
nonfinite report values fail as report_result_invalid, reason nonfinite.
133134
134135
Selected-window mode (selected-window wire v1.3, full/v1 report, no digest). Opt in
135136
with --report-policy selected-window/v1, --window-start-ms T, --window-end-ms E,
@@ -3543,12 +3544,13 @@ def fmt_utc(ms: int) -> str:
35433544
ms / 1000, tz=timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
35443545

35453546

3546-
def _num(x):
3547-
"""JSON-safe float. The engine's metric NaN convention (empty / zero
3548-
denominator -> NaN, never 0) cannot survive JSON: json.dump emits a bare
3549-
`NaN` token that a strict downstream JSON.parse (the MCP layer) rejects.
3550-
Collapse every non-finite double to null so the report stays valid JSON."""
3547+
def _num(x, *, report_metric=False):
3548+
"""Native metric NaN is the producer's explicit defined-absence sentinel.
3549+
Preserve infinity until _validate_report_result can raise the typed R.
3550+
Recorded script outputs have their separate nonfinite-to-null convention."""
35513551
f = float(x)
3552+
if report_metric:
3553+
return None if math.isnan(f) else f
35523554
return f if math.isfinite(f) else None
35533555

35543556

@@ -3562,13 +3564,15 @@ def _num(x):
35623564
def _stats_dict(s) -> dict:
35633565
"""Serialize a pf_trade_stats_t / pf_equity_stats_t ctypes struct to a dict,
35643566
keying off each field's ctype: integer counters stay ints, every double is
3565-
sanitized through _num. Driven by _fields_ so it tracks the struct verbatim;
3567+
normalized through _num's metric mode. Computed nonfinite values must
3568+
already have failed native M checks; infinities also fail the C backstop.
3569+
Driven by _fields_ so it tracks the struct verbatim;
35663570
an equity struct's two monthly ratios take their EQUITY_REPORT_KEYS."""
35673571
keys = EQUITY_REPORT_KEYS if isinstance(s, EquityStatsC) else {}
35683572
out = {}
35693573
for name, ctype in s._fields_:
35703574
v = getattr(s, name)
3571-
out[keys.get(name, name)] = _num(v) if ctype is ctypes.c_double else int(v)
3575+
out[keys.get(name, name)] = _num(v, report_metric=True) if ctype is ctypes.c_double else int(v)
35723576
return out
35733577

35743578

@@ -3613,9 +3617,18 @@ def build_report_dict(report: ReportC, ohlcv_path: Path,
36133617

36143618
cum, peak, max_dd = 0.0, 0.0, 0.0
36153619
for p in pnls:
3620+
if not math.isfinite(p):
3621+
raise RunFailure("the report result trades.pnl is not finite",
3622+
"report_result_invalid",
3623+
{"field": "trades.pnl", "reason": "nonfinite"})
36163624
cum += p
36173625
peak = max(peak, cum)
3618-
max_dd = min(max_dd, cum - peak)
3626+
drawdown = cum - peak
3627+
if not math.isfinite(cum) or not math.isfinite(drawdown):
3628+
raise RunFailure("the report result summary.max_drawdown is not finite",
3629+
"report_result_invalid",
3630+
{"field": "summary.max_drawdown", "reason": "nonfinite"})
3631+
max_dd = min(max_dd, drawdown)
36193632

36203633
# Computed trading metrics (ABI v2): all/longs/shorts trade stats + the
36213634
# equity-curve-derived block (sharpe/sortino/cagr/calmar/...). See the
@@ -3636,8 +3649,8 @@ def build_report_dict(report: ReportC, ohlcv_path: Path,
36363649
p = report.equity_curve[i]
36373650
equity_curve.append({
36383651
"time_ms": int(p.time_ms),
3639-
"equity": _num(p.equity),
3640-
"open_profit": _num(p.open_profit),
3652+
"equity": float(p.equity),
3653+
"open_profit": float(p.open_profit),
36413654
})
36423655

36433656
return {
@@ -3682,9 +3695,9 @@ def build_report_dict(report: ReportC, ohlcv_path: Path,
36823695

36833696

36843697
def _validate_report_result(out: dict) -> None:
3685-
"""Canonical trade and summary doubles have no defined non-finite absence.
3686-
Keep the failure fields as engine-owned literals; metrics, equity and
3687-
recorded outputs retain their existing _num-to-null convention."""
3698+
"""Required results are finite; only native metric absence is nullable.
3699+
Metric keys come from the producer's ctypes descriptors and alias map,
3700+
after native computation has distinguished absence from computed NaN."""
36883701
for trade in out["trades"]:
36893702
for key, field in (
36903703
("entry_price", "trades.entry_price"),
@@ -3701,6 +3714,7 @@ def _validate_report_result(out: dict) -> None:
37013714
"report_result_invalid",
37023715
{"field": field, "reason": "nonfinite"})
37033716
for key, field in (
3717+
("win_rate_pct", "summary.win_rate_pct"),
37043718
("net_pnl", "summary.net_pnl"),
37053719
("avg_trade", "summary.avg_trade"),
37063720
("best_trade", "summary.best_trade"),
@@ -3711,6 +3725,22 @@ def _validate_report_result(out: dict) -> None:
37113725
raise RunFailure(f"the report result {field} is not finite",
37123726
"report_result_invalid",
37133727
{"field": field, "reason": "nonfinite"})
3728+
for block in ("all", "longs", "shorts", "equity"):
3729+
for name, value in out["metrics"][block].items():
3730+
if value is not None and not math.isfinite(value):
3731+
metric_field = f"metrics.{block}.{name}"
3732+
raise RunFailure(f"the report result {metric_field} is not finite",
3733+
"report_result_invalid",
3734+
{"field": metric_field, "reason": "nonfinite"})
3735+
for point in out["equity_curve"]:
3736+
for key, field in (
3737+
("equity", "equity_curve.equity"),
3738+
("open_profit", "equity_curve.open_profit"),
3739+
):
3740+
if not math.isfinite(point[key]):
3741+
raise RunFailure(f"the report result {field} is not finite",
3742+
"report_result_invalid",
3743+
{"field": field, "reason": "nonfinite"})
37143744

37153745

37163746
def _request_invalid(text: str, option: str, exit_status: int = 1) -> RunFailure:

‎docs/adr/0001-kernel-adapter-boundary.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ summary of it.
8686
`NativeCoordinate::interval_index` (`market_driver.hpp:74`) is the sequential
8787
SCRIPT-bar index even when `input_tf` is finer than `script_tf`. The same
8888
script coordinate books `PyramidEntry::entry_bar_index`, trade-row indices and
89-
the inclusive bar metrics (`engine_orders.cpp:113-114`, `engine_metrics.cpp:140`).
89+
the inclusive bar metrics (`engine_orders.cpp:113-114`, `engine_metrics.cpp:157`).
9090
`NativeCoordinate::input_interval_index` (`market_driver.hpp:79`) names the
9191
input slot that supplied the point for a host that needs input cadence. Both
9292
coordinate values are folded by `hash_coordinate`
@@ -172,7 +172,7 @@ summary of it.
172172
(`guard_native_mutation` `engine_aux_security.cpp:83`).
173173
Indicators are `ta_*.cpp`, session/calendar `session_time.cpp` / `native_calendar.cpp`, FX
174174
`configure_native_fx_curve` (`native_host.hpp:1202`).
175-
- **Reporting.** `fill_report` (`engine.hpp:1859`, `src/engine_report.cpp:56`) gives a bare host
175+
- **Reporting.** `fill_report` (`engine.hpp:1859`, `src/engine_report.cpp:58`) gives a bare host
176176
trade stats, and the equity curve is a run-spec choice rather than a host chore.
177177
`NativeReportPolicy::HostRecorded` (the default) leaves the series to the host, whose
178178
`record_equity_point` / `update_equity_extremes` remain protected
@@ -181,8 +181,8 @@ summary of it.
181181
`KernelRecordedAtHostMarks` records the same series at the host's own marks.
182182
`report_open_position_at_end` books the still-open position as a mark-to-market row under
183183
`KernelRecorded`. Under `HostRecorded` an unrecorded curve still degenerates —
184-
`compute_equity_stats` answers its all-NaN value with no error (`src/engine_metrics.cpp:203`,
185-
consumed by `fill_metrics_section`, `src/engine_report.cpp:138-145`) — which is why the policy exists.
184+
`compute_equity_stats` answers its defined empty-curve metrics (`src/engine_metrics.cpp:221`,
185+
consumed by `fill_metrics_section`, `src/engine_report.cpp:203`) — which is why the policy exists.
186186
- **Reproducibility.** Two digests, one wrapping the other. `native_continuation_hash()`
187187
(`native_host.hpp:1376`) returns the consumer's event/continuation hash. **A digest names
188188
the run's inputs** (rule 2): the timezone identity enters it as the zone's *content* —
@@ -764,7 +764,7 @@ number names its symbol on this tree.
764764
| `strategy_configure_native_v1` (`c_abi.cpp:980`) versus `strategy_configure_native_ext_result_v1` (`native_c_host.cpp:4066`) | **retained legacy base plus typed additive route** | A base specification the kernel rejects still returns `-1` and latches `Failed`, preserving its frozen caller contract; a Ready misuse retains that Contract latch. The C boundary refuses arguments, a Running phase and a Completed hash hook without entering kernel configure, so a read cannot reconfigure its host. The extended result call validates before its first configure, writes `pf_native_spec_error_t` / `pf_native_spec_field_t`, and names reused-host session-key and run-number refusals; no old signature or layout moved. |
765765
| `strategy_configure_native_fx_curve_v1` (`pineforge.h:546`) | **typed additive route** | `strategy_configure_native_fx_curve_ext_v1` writes the C twin of `NativeFxCurveValidation` and its offending index. The old `-1` route remains unchanged, while a C caller can now distinguish a decreasing timestamp, non-positive rate, allocation failure or wrong phase. |
766766
| `on_native_timeframe_bar` (`native_host.hpp:885`) and `NativeTimeframeBarContext::interval` (`native_host.hpp:809`) | **query routed** | The callback signature stays frozen. `strategy_native_timeframe_bar_interval_v1` is valid only during that callback and copies the kernel calendar interval without deriving it from the delivered bar or delivery timestamp. |
767-
| `magnifier_sub_bars_total` / `magnifier_sample_ticks_total` (`engine_report.cpp:75-76`) | **kernel-owned report facts** | The generic intrabar driver already increments `NativeDriverStatistics::sub_bars_processed` / `sample_ticks_processed` for every retained path (`native_execution_consumer.cpp:8681-8693`); the kernel now publishes those same counters for bare hosts. They are report fields, not new durable state: the existing `native_continuation_hash` fold of `NativeDriverStatistics` (`native_execution_consumer.cpp:1125-1130`) sees the same values, so no hash contract moves. |
767+
| `magnifier_sub_bars_total` / `magnifier_sample_ticks_total` (`engine_report.cpp:77-78`) | **kernel-owned report facts** | The generic intrabar driver already increments `NativeDriverStatistics::sub_bars_processed` / `sample_ticks_processed` for every retained path (`native_execution_consumer.cpp:8681-8693`); the kernel now publishes those same counters for bare hosts. They are report fields, not new durable state: the existing `native_continuation_hash` fold of `NativeDriverStatistics` (`native_execution_consumer.cpp:1125-1130`) sees the same values, so no hash contract moves. |
768768
| `pf_native_lot_excursion_v1` (`native_c_api.h:1912`) | **additive fee tail** | `entry_commission` carries the C++ `ClosedLotExcursionFacts::entry_commission` in account currency. The callback table publishes a trailing layout marker so F4 callers still receive the base facts length; current callers receive the fee tail without changing `PF_NATIVE_API_VERSION`. |
769769
| `strategy_outputs_set_enabled` and the eight `strategy_outputs_*` readers (`pineforge.h`, group `pf_outputs`) | **additive runtime exports** | Recording is off by default, and nothing a run computes reads the record. The event copy is size-prefixed like `strategy_pending_order_get`, and `PF_ABI_VERSION` stays 4. The writers are C++ only in this version (the BASE-CLASS SEAMS rows of `native_c_api.h`): a C host declares no outputs, so its handle answers every reader as empty. |
770770

‎docs/design/native-feature-parity.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -156,9 +156,9 @@ Columns: **Feature** (adapter mechanism, cited) · **Native today** (`yes` / `pa
156156
| ID | Feature (adapter mechanism) | Native today | Class | Lane | Sources | Notes |
157157
|---|---|---|---|---|---|---|
158158
| RP1 | Per-lot MFE / MAE, kernel-sampled or host-owned | **yes** — sampler `apply_excursion` native_execution_consumer.cpp:3280, closing-row fold `max_runup` engine_orders.cpp:157-171; `owns_lot_excursions` native_host.hpp:1021-1025 | K+A | — | F:H1 O:I1 O:I2 S:X1 | **Disagree:** S partial + its P8 excursion-mode enum (single-source, not adopted, Appendix A.9). **Lowered by R5 lane H-THIN:** the Pine host keeps the kernel's `owns_lot_excursions()` default, so the sampler is the Pine excursion; the host model, its entry-bar masks and its caches are deleted, and the source layer's fold no longer carries them. **Measured against TradingView first (R5 lane H-MEASURE, `tests/test_e19_excursion_tape.cpp`):** the host-owned model was `ab9714be`'s, not TradingView's — where it and the kernel sampler differ the kernel's number is TradingView's (entry-bar over-mask, exit-bar over-fold after an at-open exit, a `calc_on_order_fills` scratch TradingView reports as 0 / 0); both miss a bar's open print ahead of a same-bar stop fill. On the corpus the two models differ on 341 of 431 390 trades, and of the 326 of those a TradingView export identifies, the kernel's number is TradingView's on 315 and the host's on none. The lowering moved the cells of 335 trades in 19 corpus probes: 324 now equal TradingView's, 11 moved closer, none farther; the all-in residual's own entry-bar call is the tape of the private fixture set `e19_allin_trim`. |
159-
| RP2 | Closed-trade list, trade statistics, report struct | **yes** — `fill_report` engine_report.cpp:39-85, `get_trade` engine.hpp:1792-1829, `compute_trade_stats` engine_metrics.cpp:122 | K | — | F:H2 O:I3 S:X2 | — |
159+
| RP2 | Closed-trade list, trade statistics, report struct | **yes** — `fill_report` engine_report.cpp:58-105, `get_trade` engine.hpp:1792-1829, `compute_trade_stats` engine_metrics.cpp:132 | K | — | F:H2 O:I3 S:X2 | — |
160160
| RP3 | Equity curve | **no** — `update_equity_extremes` engine.hpp:1362 / `record_equity_point` engine.hpp:1380 are protected and were called only by the adapter **Closed:** `NativeReportPolicy::KernelRecorded` native_run_spec.hpp:63 has the consumer fold the extremes and append one point per script calculation, labelled with the script interval's open; `KernelRecordedAtHostMarks` records the same series at the host's own marks (`tests/test_native_report_truth.cpp`). | K | L2 | F:H3 O:I4 S:X2 | S's "partial" folds RP2 in |
161-
| RP4 | Equity metrics (drawdown, run-up, Sharpe / Sortino, CAGR, time in market) | **no** — `compute_equity_stats` exits early on an empty curve engine_metrics.cpp:203; `fill_metrics_section` engine_report.cpp:138-145 **Closed:** with a recorded curve the drawdown / run-up walk is finite and every equity metric is computed over a real series; `report_open_position_at_end` adds the still-open position as a mark-to-market row (`tests/test_native_report_truth.cpp`). | K | L2 | F:H3 O:I5 | silent degenerate metrics, no error (O) |
161+
| RP4 | Equity metrics (drawdown, run-up, Sharpe / Sortino, CAGR, time in market) | **no** — `compute_equity_stats` exits early on an empty curve engine_metrics.cpp:221; `fill_metrics_section` engine_report.cpp:203 **Closed:** with a recorded curve the drawdown / run-up walk is finite and every equity metric is computed over a real series; `report_open_position_at_end` adds the still-open position as a mark-to-market row (`tests/test_native_report_truth.cpp`). | K | L2 | F:H3 O:I5 | silent degenerate metrics, no error (O) |
162162
| RP5 | Range-end row for a position still open at the end (`scheduler_record_range_end` pine_strategy_host.cpp:1982) | **yes, shared** — one generic producer `NativeExecutionConsumer::append_open_position_report_rows`, called by the kernel's run-end `record_open_position_report_rows` (`KernelRecorded` + `report_open_position_at_end`) and by the adapter at TradingView's mark; `report_trade_count()` engine.hpp:1831 includes the rows | K + A report shape | L2, Q6 | F:H4 O:I6 S:X2 | **Disagree:** S says kernel range-end rows exist (storage only). The ROWS are now the kernel's (§3.7 part 1); TV report shape — the equity re-mark, the extreme re-fold, the same-bar re-sort, three marks — stays A, measured in §3.7 part 2. The short-seed swaps (`project_short_seed_report_rows` pine_strategy_host.cpp:1444) stay A as well: a projection of two rows' `entry_incarnation` onto `ab9714be`'s numbering, which TradingView does not export; measured against the kernel's own lot identities by `tests/test_short_seed_report_swap.cpp` (R5 lane H-MEASURE), and pinned by `test_native_oracle_short_seed_percent_full_l2`, whose two incarnation checks fail when the swap is skipped. The swap runs only when a later close of the seed id closes a remnant; a flat collision keeps the kernel's numbering. |
163163
| RP6 | Per-trade accessors | **partial** — `get_trade` is public; the `closed_trade_*` family is protected engine.hpp:1391 **Closed:** `closed_trade_count()` / `closed_trade(i)` and `report_trade_count()` / `get_report_trade(i)` are public, and `closed_trade_close_cause(i)` engine.hpp:1849 answers the typed `execution::CloseCause` the closer recorded (`tests/test_native_report_truth.cpp`). | K | L2 | F:H6 | (single-source) |
164164
| RP7 | Live order-action stream (runner ledger / webhook) | **yes** — `stream_observe_actions_` engine_execution.cpp:1383, enabled `stream_observe_actions_` native_execution_consumer.cpp:10589, C ABI `strategy_stream_order_actions_len` c_abi.cpp:737-754 | K | — | F:H5 | (single-source) |

0 commit comments

Comments
 (0)