Skip to content

Commit efb8c7e

Browse files
luisleo526claude
andcommitted
o1w-contract round 2: the one O(1) report contract — modes, pages, envelope (supervisor commit)
Executor result of lane core-cr150-o1w-contract (Codex gpt-6.1-sol), committed by the supervisor. Round 2 rewrites the round-1 draft onto the final names: include/pineforge/report_mode.h (ReportMode FULL/SUMMARY_ONLY/STREAMED, ReportEnvelopeV1, fill_report_envelope_v1, set_report_mode_v1) and include/pineforge/report_pages.hpp (TradeSourceV1, EquityPointSourceV1, detached row/point visitors, HeldEquityPointV1), with engine.hpp reader acquisition, metrics_stream.hpp point-provider overloads, the contract document docs/design/o1-report-contract.md, report-schema.md, a C++ contract test, a Python unsupported-mode test and four-row/four-point fixtures. The a1ctr draft's report_output.hpp is folded in by renaming (appendix in the contract document records every old->new name and every omitted draft-only option). ABI counts unchanged (104 public declarations / 84 implementations; native-C 43/43). NOT VERIFIED: no compile, no test, no corpus ran on this tree. Lab rows owed: test_o1_report_contract, test_o1_report_mode_unsupported (fail-before on the unchanged tree, green after), doc-anchors re-pin, then the full proof. Co-Authored-By: Claude Code <[email protected]>
1 parent 583c221 commit efb8c7e

15 files changed

Lines changed: 1793 additions & 16 deletions

‎docs/design/o1-report-contract.md‎

Lines changed: 498 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/pages/report-schema.md‎

Lines changed: 176 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,10 @@
22

33
@tableofcontents
44

5-
`pf_report_t` is the single output of every backtest. This page maps
6-
each field to its meaning and units.
5+
`pf_report_t` is the legacy backtest payload. This page maps each field to its
6+
meaning and units, and defines the opt-in `pineforge-report/v1` envelope for
7+
FULL, SUMMARY_ONLY and STREAMED reports. The envelope and native methods are
8+
declarations in this slice; their presence does not advertise runtime support.
79

810
@see #pf_report_t for the verbatim struct declaration.
911

@@ -17,7 +19,9 @@ The `--capabilities-json` probe's
1719
`capabilities.selected_window_contract_revisions` is `["1.3-parity"]` for a
1820
supported runtime and `[]` otherwise. The declaration lives in that capability
1921
record; the selected-window wire version remains 1, the phase version 1 and the
20-
fingerprint/schema version 2.
22+
legacy selected fingerprint/schema version 2. Explicit report modes use the
23+
independent envelope version 1 and fingerprint/schema version 3 described
24+
below.
2125

2226
## Top-level layout
2327

@@ -84,13 +88,153 @@ flag; a v2 reader would misindex the trades array. **ABI version 4**
8488
appends `pf_report_t::broker_state_hash` / `broker_state_hash_len`; a v3
8589
reader's struct is 16 bytes too small.
8690

91+
## Versioned report modes {#report_modes_v1}
92+
93+
The full contract is
94+
[O1 report contract](../design/o1-report-contract.md). Include
95+
`<pineforge/report_mode.h>` for the C-compatible data declarations. Native C++
96+
clients use `ReportMode`, `ReportEnvelopeV1`,
97+
`BacktestEngine::set_report_mode_v1` and
98+
`BacktestEngine::fill_report_envelope_v1`. The methods are declarations only
99+
until the runtime implementation lands. C/ctypes clients require an explicitly
100+
negotiated provider bridge; there is no new runtime C export in this slice.
101+
102+
| Mode | Metrics and summary scalars | Resident sequences | External sequences |
103+
| --- | --- | --- | --- |
104+
| FULL (`0`) | Existing values/bits | Existing owning arrays | None through the explicit FULL envelope |
105+
| SUMMARY_ONLY (`1`) | Bit-identical to FULL, including all four metric blocks | None | No captured report sequences or private curve tape |
106+
| STREAMED (`2`) | Same values/bits | None | Complete enabled sequences, in existing report order, through a bounded sink |
107+
108+
Legacy calls, including the existing sink-v1 route, keep their current
109+
behavior. Explicit FULL conflicts with an installed sink; STREAMED requires
110+
one; SUMMARY_ONLY conflicts with sequence-sink capture. FULL is an owning-array
111+
compatibility action and makes no fixed-memory claim. SUMMARY_ONLY must not
112+
allocate full arrays and delete them after reporting.
113+
114+
The new data layout wraps the ABI-4 payload rather than growing it:
115+
116+
```c
117+
typedef struct pf_report_sequence_v1 {
118+
int64_t logical_count;
119+
uint32_t captured; /* 0/1; an empty captured sequence still has 1 */
120+
uint32_t external; /* 0/1; implies captured == 1 */
121+
} pf_report_sequence_v1;
122+
123+
typedef struct pf_report_metadata_v1 {
124+
uint32_t struct_size; /* sizeof(pf_report_metadata_v1) == 128 */
125+
uint32_t version; /* PF_REPORT_ENVELOPE_VERSION == 1 */
126+
pf_report_mode_v1_t mode; /* fixed-width uint32_t */
127+
uint32_t complete; /* 0/1; independent of every logical count */
128+
pf_report_sequence_v1 trades;
129+
pf_report_sequence_v1 equity_curve;
130+
pf_report_sequence_v1 broker_state_hash;
131+
pf_report_sequence_v1 trace;
132+
pf_report_sequence_v1 trace_names;
133+
pf_report_sequence_v1 output_rows;
134+
pf_report_sequence_v1 output_events;
135+
} pf_report_metadata_v1;
136+
137+
typedef struct pf_report_envelope_v1 {
138+
pf_report_metadata_v1 metadata;
139+
pf_report_t report; /* unchanged ABI-4 payload at offset 128 */
140+
} pf_report_envelope_v1;
141+
```
142+
143+
Initialize metadata's `struct_size` and `version` before a versioned native
144+
read. `PF_ABI_VERSION` stays 4: passing an envelope to a legacy report function
145+
is invalid. Free the embedded payload with the existing report free operation;
146+
the caller owns the envelope. Bounded per-definition security diagnostics
147+
remain in the payload in all modes.
148+
149+
Each `logical_count` counts final records from the authoritative pass once,
150+
regardless of capture. In SUMMARY_ONLY every sequence has `captured=0` and
151+
`external=0`. In FULL an enabled sequence has `captured=1, external=0`; in
152+
STREAMED it has `captured=1, external=1`. Disabled optional sources have
153+
count/flags zero. Thus omitted equity and a successfully captured empty curve
154+
are distinguishable. `complete=1` additionally requires successful execution
155+
and report validation, matched replay witnesses where needed, and a complete
156+
STREAMED sink receipt. Lengths and counts never prove completion.
157+
158+
`report.total_trades` and `metadata.trades.logical_count` both contain the
159+
logical T, including terminal open-at-end rows. `report.trades_len` is the
160+
resident length; it is zero in SUMMARY_ONLY and STREAMED even when T is
161+
positive. Likewise `metadata.equity_curve.logical_count` counts logical
162+
presented points while `report.equity_curve_len` describes only the resident
163+
array. The latter is zero in SUMMARY_ONLY and STREAMED. Ordinary completed
164+
KernelRecorded reports have one logical point per script bar; selected
165+
reports have M+1, including the anchor.
166+
167+
SUMMARY_ONLY preserves Sharpe/Sortino through an exact second ordered replay
168+
when the existing return-count guards require it. Publication checks the
169+
final trade count, exact ordered trade witness, point count, first/last
170+
coordinates and final continuation/hash witnesses. Replay repeats no outward
171+
observer, alert or sink side effects. Reports read a frozen summary rather
172+
than rerunning callbacks. Nonreplayable hosts/presenters require an exact
173+
adapter or an explicit refusal; no metric may be silently omitted.
174+
175+
Transport selection is `--report-mode FULL|SUMMARY_ONLY|STREAMED`. A missing
176+
provider must refuse before execution with `report_mode_unsupported`, args
177+
`{"mode":"SUMMARY_ONLY","contract":"pineforge-report/v1"}` for that request,
178+
and diagnostic `unsupported report mode: SUMMARY_ONLY`. It must never return
179+
FULL, accept an empty legacy array as evidence, or ignore the requested mode.
180+
181+
Explicit-mode JSON includes `report_contract: "pineforge-report/v1"` and
182+
`report_capture`, containing `version`, the mode name, `complete`, and all
183+
seven sequence objects above. Flags become JSON booleans; `struct_size` is
184+
C-only. Existing metric keys, including `sharpe_tv` / `sortino_tv`, remain.
185+
Empty arrays in SUMMARY_ONLY always accompany this explicit capture metadata.
186+
STREAMED JSON may serialize complete sequences through bounded iterators.
187+
188+
Explicit modes also add `report_contract`, `report_mode`, and
189+
`sequence_capture` to `applied_runtime` and fingerprinted `provenance.runtime`.
190+
`sequence_capture` is false for SUMMARY_ONLY and true for FULL/STREAMED.
191+
Both `fingerprint.version` and `fingerprint.provenance.schema_version` become
192+
**3**. These policy/version bytes change the canonical provenance token and
193+
digest; legacy fingerprint byte identity is not promised. Legacy invocations
194+
keep their existing envelope/fingerprint behavior. Execution identity and
195+
broker/source hash leaves retain their existing definitions.
196+
197+
### Value/page/visitor readers {#report_readers_v1}
198+
199+
`<pineforge/report_pages.hpp>` declares the single C++ reader family in
200+
`pineforge::report`; it uses the modes/envelope above. `TradeSourceV1::at`
201+
returns an exact `TradeRowV1` value. `read_page` copies into caller-owned
202+
bounded buffers; `visit` walks a logical range forward and `visit_backward`
203+
walks the same range newest logical row first, including tied/nonmonotonic
204+
exit times. `TradeTextV1` selects IDs/comments for byte-exact bounded reads.
205+
`EquityPointSourceV1` provides value/index/page/ordered-forward point reads.
206+
Callbacks receive detached values; no API returns a reference or pointer into
207+
a resident page that may be retired. Invalid ranges/buffers and backing
208+
failures fail explicitly.
209+
210+
Native engine accessors return caller-owned reader adapters by value:
211+
`completed_trade_source_v1()` covers semantic completed history in every
212+
mode, excluding the range-end suffix; `report_trade_source_v1()` covers the
213+
whole captured final report; `report_equity_source_v1()` covers captured
214+
final presented points, including a selected anchor. Report readers require
215+
validated completion. SUMMARY_ONLY rejects the two omitted report sources
216+
explicitly rather than returning fictitious empty readers.
217+
218+
`held_report_equity_point_v1()` returns one final `HeldEquityPointV1` by value
219+
in every mode. `present=false` means zero logical points; otherwise `index`
220+
is the last logical index and `value` is the exact final presented point.
221+
This held value does not change SUMMARY_ONLY's `captured=0` or NULL/0 curve.
222+
During execution the metrics owner holds the editable tail and pre-tail
223+
fold snapshot until presentation finalizes it; summary/replay passes use
224+
the same point visitor without creating a private curve tape.
225+
226+
The [reader contract and old-draft name mapping](../design/o1-report-contract.md)
227+
define lifetimes, half-open ranges, backward traversal, cancellation and
228+
errors. Reader declarations do not establish runtime support and add no
229+
C export, mode or envelope-version change.
230+
87231
## Trade fields
88232

89233
| Field | Type | Meaning |
90234
| --- | --- | --- |
91-
| `total_trades` | `int` | Closed-trade count. Always equal to `trades_len`. |
92-
| `trades` | `pf_trade_t*` | Heap array, ordered by exit time ascending. |
93-
| `trades_len` | `int` | Length of `trades`. |
235+
| `total_trades` | `int` | Logical closed-trade total, including range-end rows. Equals `trades_len` only on the owning-array route; remains T when SUMMARY_ONLY, STREAMED or a legacy sink returns no resident rows. |
236+
| `trades` | `pf_trade_t*` | Owning array in report order: script-driven closed rows, then report-only range-end rows. NULL in SUMMARY_ONLY/STREAMED. |
237+
| `trades_len` | `int` | Resident array length, independent of the logical total. |
94238
| `net_profit` | `double` | Sum of all closed-trade PnL in account currency. |
95239

96240
### pf_trade_t
@@ -188,6 +332,10 @@ Populated only when:
188332
1. The source script contains `// @pf-trace name=expr` pragmas, **and**
189333
2. #strategy_set_trace_enabled was called with `1` before the run.
190334

335+
In the explicit envelope, FULL returns the resident records, SUMMARY_ONLY
336+
omits them, and STREAMED publishes them externally. The trace/name logical
337+
counts and capture flags remain explicit in metadata.
338+
191339
| Field | Meaning |
192340
| --- | --- |
193341
| `trace[i].timestamp` | Bar timestamp (Unix ms). |
@@ -201,9 +349,11 @@ buffer, no formatting.
201349

202350
## Metrics (ABI v2)
203351

204-
`pf_report_t::metrics` is a `pf_metrics_t` — four embedded blocks
205-
computed at report time. The complete per-metric reference (definitions,
206-
units, NaN rules, validation status) lives on @ref metrics.
352+
`pf_report_t::metrics` is a `pf_metrics_t` — four embedded blocks.
353+
Legacy reports compute them at report time; SUMMARY_ONLY returns the frozen,
354+
validated metrics without a resident curve or trade array. All 768 bytes,
355+
including the equity-derived ratios, must match FULL. The complete per-metric
356+
reference (definitions, units, NaN rules, validation status) lives on @ref metrics.
207357

208358
| Block | Type | Scope |
209359
| --- | --- | --- |
@@ -269,13 +419,21 @@ typedef struct pf_equity_point_s {
269419
} pf_equity_point_t;
270420
```
271421

272-
`equity_curve_len` equals `script_bars_processed` on a completed
273-
`KernelRecorded` run (an exception, cooperative abort or other failure
274-
mid-run can truncate the recorded prefix — check `strategy_get_last_error`). A
422+
On the ordinary legacy/FULL owning-array route, `equity_curve_len` equals
423+
`script_bars_processed` on a completed `KernelRecorded` run (an exception,
424+
cooperative abort or other failure mid-run can truncate the recorded prefix —
425+
check `strategy_get_last_error`). A
275426
bare native host using the default `HostRecorded` policy has no kernel-owned
276427
curve. The array is heap-allocated and freed by #report_free. Note the length
277428
field is `int64_t`, not `int`.
278429

430+
SUMMARY_ONLY and STREAMED set the resident pointer/length to NULL/0 while
431+
`metadata.equity_curve.logical_count` records the presented logical point
432+
count. SUMMARY_ONLY explicitly sets `captured=0`; STREAMED sets
433+
`captured=1, external=1` for an enabled curve. Selected M+1 counts include
434+
the anchor. Neither an empty array nor the logical count certifies success:
435+
inspect completion/status and the capture flags.
436+
279437
## Recorded outputs (outside the report)
280438

281439
What a module records besides its trades -- values kept per bar, values kept
@@ -284,6 +442,12 @@ once per run, events with an optional message -- is not part of
284442
through the `strategy_outputs_*` functions (@ref pf_outputs) after a batch, or
285443
between a stream's inputs; the record's rules are in `docs/outputs.md`.
286444

445+
That describes the legacy/FULL route. In SUMMARY_ONLY the versioned contract
446+
keeps bounded manifest/constants/hlines but captures no row-time, series or
447+
event history; logical row/event counts appear in envelope metadata. STREAMED
448+
publishes the enabled histories externally. Neither mode promises that the
449+
legacy owning output accessors provide an arbitrarily sized resident array.
450+
287451
`docker/run_json.py --outputs` writes the record into its JSON report as an
288452
`outputs` key, just before `fingerprint`, and marks the run with
289453
`"outputs": true` in `applied_runtime` and in the fingerprint's

‎include/pineforge/engine.hpp‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,8 @@
4444
#define PINEFORGE_NO_STRATEGY_DECLS
4545
// Angle-bracket form is the installed public path (deliberate).
4646
#include <pineforge/pineforge.h>
47+
#include <pineforge/report_mode.h>
48+
#include <pineforge/report_pages.hpp>
4749
#include <pineforge/report_sink_writer.hpp>
4850
// Generated script lifecycle users must be rebuilt against this runtime.
4951
// These internal C++ capabilities change no public C POD or C signature.
@@ -298,6 +300,27 @@ struct ReportC {
298300
int64_t broker_state_hash_len;
299301
};
300302

303+
// O1 report envelope v1. Data and method declarations do not advertise
304+
// runtime support; the legacy ReportC layout remains ABI 4.
305+
enum class ReportMode : std::uint32_t {
306+
FULL = PF_REPORT_MODE_FULL,
307+
SUMMARY_ONLY = PF_REPORT_MODE_SUMMARY_ONLY,
308+
STREAMED = PF_REPORT_MODE_STREAMED
309+
};
310+
311+
struct ReportEnvelopeV1 {
312+
pf_report_metadata_v1 metadata;
313+
ReportC report;
314+
};
315+
316+
static_assert(sizeof(ReportC) == sizeof(pf_report_t),
317+
"legacy C and C++ report sizes must match");
318+
static_assert(offsetof(ReportEnvelopeV1, report)
319+
== offsetof(pf_report_envelope_v1, report),
320+
"versioned report payload offsets must match");
321+
static_assert(sizeof(ReportEnvelopeV1) == sizeof(pf_report_envelope_v1),
322+
"versioned C and C++ report sizes must match");
323+
301324
using ExitLegLifecycle = exit_legs::Lifecycle;
302325

303326
// default_qty_type constants (matches TradingView)
@@ -1860,6 +1883,36 @@ class BacktestEngine {
18601883
void fill_report(ReportC* out) const;
18611884
static void free_report(ReportC* report);
18621885

1886+
// Declarations only; implementation belongs to the O1 runtime lanes.
1887+
// Select before execution; inspect the PF_REPORT_MODE_* return status.
1888+
// Rejection is atomic. SUMMARY_ONLY must never fall back to FULL.
1889+
int set_report_mode_v1(ReportMode mode);
1890+
// Initialize metadata.struct_size/version before reading. A successful
1891+
// read copies a frozen report without replay or outward side effects.
1892+
// Release the owned payload with free_report(&out->report).
1893+
int fill_report_envelope_v1(ReportEnvelopeV1* out) const;
1894+
1895+
// O1 reader declarations only. Returned unique_ptr values own fixed-size
1896+
// reader adapters, never a row/curve array or a resident page. Adapters
1897+
// borrow this engine until its next execution/config mutation/destruction;
1898+
// detached row/point/caller-buffer values survive adapter/page retirement.
1899+
// Successful acquisition is non-null, including for a captured empty
1900+
// sequence; absence/unavailability is an error, never a null sentinel.
1901+
// Exact semantic completed rows exclude report-only range-end rows and
1902+
// are readable in every mode at a committed script callback boundary.
1903+
std::unique_ptr<const report::TradeSourceV1> completed_trade_source_v1() const;
1904+
// Final report order includes range-end rows; equity includes the selected
1905+
// anchor. FULL/STREAMED sources read captured output through bounded pages.
1906+
// SUMMARY_ONLY throws std::logic_error for these two omitted sequences,
1907+
// never a fictitious empty source or an implicit FULL materialization.
1908+
// An uncaptured/disabled equity source also throws in FULL/STREAMED.
1909+
// Reading a report source/tail before validated completion also throws.
1910+
std::unique_ptr<const report::TradeSourceV1> report_trade_source_v1() const;
1911+
std::unique_ptr<const report::EquityPointSourceV1> report_equity_source_v1() const;
1912+
// Single final presented point, by value, in every mode. An empty logical
1913+
// curve has present=false; this never changes the sequence capture flags.
1914+
report::HeldEquityPointV1 held_report_equity_point_v1() const;
1915+
18631916
// Returns the error message captured by the most recent run() if it
18641917
// failed, or an empty string if the run completed normally. Cleared at
18651918
// the start of every run(). The C ABI exposes this via

‎include/pineforge/metrics_stream.hpp‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
// with scratch storage in 3.8:902-909 and selected observations in 3.6:745-773.
55
// Existing array entry points in metrics.hpp remain unchanged.
66
#include <pineforge/metrics.hpp>
7+
#include <pineforge/report_pages.hpp>
78

89
#include <cstdint>
910
#include <memory>
@@ -130,5 +131,28 @@ pf_equity_stats_t compute_equity_stats(
130131
std::int64_t bars_in_market, double net_profit,
131132
std::int64_t observation_count);
132133

134+
// O1 report-point provider overloads: declarations only, metrics-lane bodies.
135+
// FULL/STREAMED can reread captured final points in the same five ordered
136+
// walks through visit()/bounded pages. n counts points, observation_count
137+
// counts observations (selected M+1 points, M observations). Preserve every
138+
// formula, float operation grouping, calendar/NaN guard and presented value.
139+
// Source/read failures propagate; never adapt by materializing an array.
140+
//
141+
// SUMMARY_ONLY does not create a captured provider or a private curve tape.
142+
// Its fixed execution replay passes use the same by-value EquityVisitorV1
143+
// inputs and held-point rule, then freeze the witnessed metrics for envelope
144+
// reads. Release the non-recursive timezone guard before execution dispatch.
145+
pf_equity_stats_t compute_equity_stats(
146+
const report::EquityPointSourceV1& curve, std::int64_t n,
147+
double initial_capital, const std::string& chart_tz,
148+
double first_open, double last_close, std::int64_t bars_in_market,
149+
double net_profit);
150+
151+
pf_equity_stats_t compute_equity_stats(
152+
const report::EquityPointSourceV1& curve, std::int64_t n,
153+
double initial_capital, const std::string& chart_tz,
154+
double first_open, double last_close, std::int64_t bars_in_market,
155+
double net_profit, std::int64_t observation_count);
156+
133157
} // namespace metrics
134158
} // namespace pineforge

0 commit comments

Comments
 (0)