Skip to content

Commit bf024f1

Browse files
luisleo526claude
andcommitted
o1w-tradeout round 2 (DeepSeek authoring): flat range-end rows and streamed trade output R12/R13 (supervisor commit)
Executor result of lane core-cr150-o1w-tradeout (DeepSeek authoring; the Codex round exited 129 with no changes on disk), committed by the supervisor. R12 replaces the resident std::vector<Trade> range-end rows with OpenReportRowSource, which keeps only the terminal mark and rebuilds each row through the same builder, one at a time, in lot order, with a borrowed-pointer materialize() kept only for the legacy C string accessors. R13 makes the summary/streamed trade sections report logical counts with null arrays and bit-identical folds, streams range-end rows in the sink, implements set_report_mode_v1 and fill_report_envelope_v1, and carries include/pineforge/report_mode.h byte-identical to the contract lane's file. NOT VERIFIED: no compile, no fail-before run, no test, no corpus ran on this tree. The broker-state waiver text stays on range_end_trades_ (same name). Lab rows owed: test_open_report_rows_flat, test_open_report_rows_retention_flat, test_report_trades_summary_flat, test_range_end_close, test_range_end_close_l4d, test_accounting_reconciliation, and the doc-anchors re-pin (the lane reports the full shift map). Co-Authored-By: Claude Code <[email protected]>
1 parent 583c221 commit bf024f1

14 files changed

Lines changed: 1335 additions & 52 deletions

‎include/pineforge/engine.hpp‎

Lines changed: 148 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,16 @@
11
#pragma once
22
#include <atomic>
3+
#include <deque>
34
#include <vector>
45
#include <string>
56
#include <cstddef>
67
#include <cstdint>
78
#include <cmath>
89
#include <ctime>
10+
#include <iterator>
911
#include <limits>
1012
#include <memory>
13+
#include <utility>
1114
#include <set>
1215
#include <algorithm>
1316
#include <unordered_map>
@@ -44,6 +47,7 @@
4447
#define PINEFORGE_NO_STRATEGY_DECLS
4548
// Angle-bracket form is the installed public path (deliberate).
4649
#include <pineforge/pineforge.h>
50+
#include <pineforge/report_mode.h>
4751
#include <pineforge/report_sink_writer.hpp>
4852
// Generated script lifecycle users must be rebuilt against this runtime.
4953
// These internal C++ capabilities change no public C POD or C signature.
@@ -298,6 +302,117 @@ struct ReportC {
298302
int64_t broker_state_hash_len;
299303
};
300304

305+
// O1 report envelope v1. Data and method declarations do not advertise
306+
// runtime support; the legacy ReportC layout remains ABI 4.
307+
enum class ReportMode : std::uint32_t {
308+
FULL = PF_REPORT_MODE_FULL,
309+
SUMMARY_ONLY = PF_REPORT_MODE_SUMMARY_ONLY,
310+
STREAMED = PF_REPORT_MODE_STREAMED
311+
};
312+
313+
struct ReportEnvelopeV1 {
314+
pf_report_metadata_v1 metadata;
315+
ReportC report;
316+
};
317+
318+
static_assert(sizeof(ReportC) == sizeof(pf_report_t),
319+
"legacy C and C++ report sizes must match");
320+
static_assert(offsetof(ReportEnvelopeV1, report)
321+
== offsetof(pf_report_envelope_v1, report),
322+
"versioned report payload offsets must match");
323+
static_assert(sizeof(ReportEnvelopeV1) == sizeof(pf_report_envelope_v1),
324+
"versioned C and C++ report sizes must match");
325+
326+
// Reopened here so the row source and the engine it borrows live in the same
327+
// ABI epoch (the inline namespace is the same one that holds BacktestEngine).
328+
inline namespace engine_script_run_v19 {
329+
class BacktestEngine;
330+
331+
// O1 R12: the terminal range-end row source.
332+
//
333+
// A run that ends with P physical lots open reports one row per lot. A row is
334+
// a pure function of (lot, terminal mark, account FX) through the same
335+
// non-mutating builder a full close uses, so this object keeps only the mark
336+
// descriptor and the already-folded row PnL; every reader builds one row at a
337+
// time, in lot order. Resident storage is constant in P: no row array and no
338+
// per-lot string copy.
339+
//
340+
// materialize() is the legacy borrowed-pointer boundary. Only the C accessors
341+
// that hand out a `const Trade&` or a string pointer (get_report_trade and its
342+
// callers) need a row that outlives the call; it keeps exactly those rows, as
343+
// the old vector did, and is documented non-flat. Report construction, folds
344+
// and sink emission never call it.
345+
class OpenReportRowSource {
346+
public:
347+
OpenReportRowSource() = default;
348+
explicit OpenReportRowSource(const BacktestEngine* owner) : owner_(owner) {}
349+
OpenReportRowSource(const OpenReportRowSource&) = default;
350+
OpenReportRowSource& operator=(const OpenReportRowSource&) = default;
351+
// A new run's terminal mark replaces the previous one; the C accessors'
352+
// retained rows belong to the replaced generation.
353+
void clear() noexcept;
354+
void begin(double mark_price, std::int64_t mark_time_ms, int interval_index,
355+
double account_fx, bool was_long, std::size_t count) noexcept;
356+
// The exact left fold append_open_position_report_rows returned.
357+
void set_pnl(double pnl) noexcept { pnl_ = pnl; }
358+
bool empty() const noexcept { return size() == 0; }
359+
std::size_t size() const noexcept { return present_ ? count_ : synthetic_.size(); }
360+
double pnl() const noexcept { return pnl_; }
361+
// One detached row, built exactly as the range-end append builds it.
362+
Trade row(std::size_t index) const;
363+
// Legacy borrowed-pointer materialization, for the C string accessors and
364+
// for tests that seed a row with no live lot.
365+
const Trade& materialize(std::size_t index) const;
366+
const Trade& operator[](std::size_t index) const { return materialize(index); }
367+
// Reference-oracle seeding only: a synthetic report row with no live lot.
368+
// Production writers pin a terminal mark instead; the two never coexist
369+
// because begin() replaces the generation.
370+
void push_back(const Trade& row) { synthetic_.push_back(row); pnl_ += row.pnl; }
371+
void push_back(Trade&& row) { pnl_ += row.pnl; synthetic_.push_back(std::move(row)); }
372+
class const_iterator {
373+
public:
374+
using iterator_category = std::forward_iterator_tag;
375+
using value_type = Trade;
376+
using difference_type = std::ptrdiff_t;
377+
using reference = Trade;
378+
using pointer = void;
379+
const_iterator() = default;
380+
Trade operator*() const { return source_->row(index_); }
381+
const_iterator& operator++() { ++index_; return *this; }
382+
const_iterator operator++(int) { auto old = *this; ++*this; return old; }
383+
bool operator==(const const_iterator& other) const {
384+
return source_ == other.source_ && index_ == other.index_;
385+
}
386+
bool operator!=(const const_iterator& other) const { return !(*this == other); }
387+
private:
388+
friend class OpenReportRowSource;
389+
const_iterator(const OpenReportRowSource* source, std::size_t index)
390+
: source_(source), index_(index) {}
391+
const OpenReportRowSource* source_ = nullptr;
392+
std::size_t index_ = 0;
393+
};
394+
const_iterator begin() const { return const_iterator(this, 0); }
395+
const_iterator end() const { return const_iterator(this, size()); }
396+
private:
397+
const BacktestEngine* owner_ = nullptr;
398+
bool present_ = false;
399+
bool was_long_ = false;
400+
double mark_price_ = 0.0;
401+
std::int64_t mark_time_ms_ = 0;
402+
int interval_index_ = 0;
403+
double account_fx_ = 1.0;
404+
std::size_t count_ = 0;
405+
double pnl_ = 0.0;
406+
// A deque gives every retained row a stable address, so a reference a C
407+
// accessor handed out survives the next row's materialization, as a
408+
// reference into the old vector did.
409+
mutable std::deque<Trade> materialized_;
410+
// Reference-oracle rows with no live lot (tests only; production leaves
411+
// this empty).
412+
std::vector<Trade> synthetic_;
413+
};
414+
} // inline namespace engine_script_run_v19
415+
301416
using ExitLegLifecycle = exit_legs::Lifecycle;
302417

303418
// default_qty_type constants (matches TradingView)
@@ -388,6 +503,9 @@ class BacktestEngine {
388503
// behaviour changed for this.
389504
friend class NativeExecutionConsumer;
390505
friend class NativeStrategyHost;
506+
// The terminal range-end row source rebuilds rows through the engine's
507+
// non-mutating close builder and reads the live lots.
508+
friend class OpenReportRowSource;
391509
struct NativeConsumerBindTag { explicit NativeConsumerBindTag() = default; };
392510
explicit BacktestEngine(NativeConsumerBindTag);
393511
IExecutionConsumer& execution_consumer();
@@ -550,7 +668,8 @@ class BacktestEngine {
550668
// spec asks for them (NativeRunSpec::report_open_position_at_end). They
551669
// are merged behind trades_ by fill_trades_section and never enter
552670
// trades_, the realized sums, or the live position (a stream continues it).
553-
std::vector<Trade> range_end_trades_;
671+
// R12: the source stores only the terminal mark; see OpenReportRowSource.
672+
OpenReportRowSource range_end_trades_{this};
554673

555674
// Best favorable price is updated by the physical open/add helpers for
556675
// every host. Source policy may consume the same value through
@@ -1832,11 +1951,18 @@ class BacktestEngine {
18321951
int report_trade_count() const {
18331952
return (int)(trades_.size() + range_end_trades_.size());
18341953
}
1954+
// A range-end row is rebuilt from its lot on every read; materialize()
1955+
// keeps the returned reference alive exactly as the old row vector did
1956+
// (the legacy C string boundary). Numeric C accessors copy at once.
18351957
const Trade& get_report_trade(int i) const {
18361958
const int n_closed = (int)trades_.size();
18371959
return i < n_closed ? trades_[(size_t)i]
1838-
: range_end_trades_[(size_t)(i - n_closed)];
1960+
: range_end_trades_.materialize((std::size_t)(i - n_closed));
18391961
}
1962+
// Why a report row closed, from the row's own recorded facts; the report
1963+
// index form below adds the row-space bounds. Kept together so a caller
1964+
// holding a row value does not need a reference into the row space.
1965+
static int classify_report_close_cause(const Trade& row);
18401966

18411967
// ABI v4 live-runtime surface (task 9): classify why a REPORT-row
18421968
// closed trade exited -- report-row scope (spans trades_ then
@@ -1860,6 +1986,20 @@ class BacktestEngine {
18601986
void fill_report(ReportC* out) const;
18611987
static void free_report(ReportC* report);
18621988

1989+
// O1 report envelope v1 (include/pineforge/report_mode.h). Select before
1990+
// execution; inspect the returned status constant (PF_REPORT_MODE_ACCEPTED
1991+
// on success). Rejection is atomic. SUMMARY_ONLY must never fall back to
1992+
// FULL.
1993+
//
1994+
// SUMMARY_ONLY and STREAMED keep the logical trade total and the exact
1995+
// folds, and return no resident TradeC array: trade rows are rebuilt one
1996+
// at a time from the closed ledger and the terminal lot mark.
1997+
int set_report_mode_v1(ReportMode mode);
1998+
// Initialize metadata.struct_size/version before reading. A successful
1999+
// read copies a frozen report without replay or outward side effects.
2000+
// Release the owned payload with free_report(&out->report).
2001+
int fill_report_envelope_v1(ReportEnvelopeV1* out) const;
2002+
18632003
// Returns the error message captured by the most recent run() if it
18642004
// failed, or an empty string if the run completed normally. Cleared at
18652005
// the start of every run(). The C ABI exposes this via
@@ -2331,6 +2471,12 @@ class BacktestEngine {
23312471
// @broker-state begin
23322472
mutable ReportSinkState report_sink_state_;
23332473
// @broker-state end
2474+
// O1 capture policy for reports built after it is set. Deliberately
2475+
// outside the broker-state region: it changes no fill, fold, hash leaf or
2476+
// continuation digest (two runs over the same input with different report
2477+
// modes must hash identically), and reset_run_state leaves it in place
2478+
// like the sink descriptor.
2479+
ReportMode report_mode_ = ReportMode::FULL;
23342480

23352481
class ReportSinkCallScope {
23362482
ReportSinkState& state;

‎include/pineforge/report_mode.h‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* O1 report contract v1: data declarations only, no runtime feature probe.
5+
* See docs/design/o1-report-contract.md. Including this header does not
6+
* establish provider support for SUMMARY_ONLY or STREAMED.
7+
* C++ row/point visitors are declared in the sibling report_pages.hpp;
8+
* they use these modes/metadata and introduce no second mode vocabulary.
9+
*/
10+
#ifndef PINEFORGE_REPORT_MODE_H
11+
#define PINEFORGE_REPORT_MODE_H
12+
13+
#include <pineforge/pineforge.h>
14+
15+
#ifdef __cplusplus
16+
extern "C" {
17+
#endif
18+
19+
/** Version of the opt-in report envelope; independent of PF_ABI_VERSION.
20+
* The legacy pf_report_t layout and all existing function signatures stay
21+
* unchanged. No runtime C export is introduced by this header. */
22+
#define PF_REPORT_ENVELOPE_VERSION 1
23+
24+
/** Fixed-width mode word, including for consumers using short C enums. */
25+
typedef uint32_t pf_report_mode_v1_t;
26+
enum {
27+
PF_REPORT_MODE_FULL = 0, /**< Legacy owning report sequences. */
28+
PF_REPORT_MODE_SUMMARY_ONLY = 1, /**< Exact metrics; no captured sequences. */
29+
PF_REPORT_MODE_STREAMED = 2 /**< Complete sequences in an external sink. */
30+
};
31+
32+
/** Setter/read outcomes. Rejection leaves configuration unchanged; invalid
33+
* envelope descriptors also leave the destination unchanged. */
34+
enum {
35+
PF_REPORT_MODE_ACCEPTED = 0,
36+
PF_REPORT_MODE_INVALID = -1,
37+
PF_REPORT_MODE_UNSUPPORTED = -2,
38+
PF_REPORT_MODE_BUSY = -3,
39+
PF_REPORT_MODE_CONFLICT = -4,
40+
PF_REPORT_MODE_REPLAY_UNAVAILABLE = -5
41+
};
42+
43+
/** Logical cardinality and explicit capture policy for one sequence.
44+
* Both flags are 0 or 1; external implies captured. An enabled, captured
45+
* empty sequence has captured=1 and logical_count=0. SUMMARY_ONLY has both
46+
* flags 0 even when logical_count is positive. These fields do not certify
47+
* run completion. Counts refer to the authoritative pass exactly once. */
48+
typedef struct pf_report_sequence_v1 {
49+
int64_t logical_count;
50+
uint32_t captured;
51+
uint32_t external;
52+
} pf_report_sequence_v1;
53+
54+
/** Fixed-size report metadata. Initialize struct_size and version before a
55+
* native envelope read. complete is 1 only after the current attempt and
56+
* all required replay/receipt validation succeed.
57+
*
58+
* The seven logical counts are independent of resident array lengths.
59+
* Optional disabled sources have count/flags 0. Output rows/events remain
60+
* outside pf_report_t; their capture policy is stated here for consumers. */
61+
typedef struct pf_report_metadata_v1 {
62+
uint32_t struct_size;
63+
uint32_t version;
64+
pf_report_mode_v1_t mode;
65+
uint32_t complete;
66+
pf_report_sequence_v1 trades;
67+
pf_report_sequence_v1 equity_curve;
68+
pf_report_sequence_v1 broker_state_hash;
69+
pf_report_sequence_v1 trace;
70+
pf_report_sequence_v1 trace_names;
71+
pf_report_sequence_v1 output_rows;
72+
pf_report_sequence_v1 output_events;
73+
} pf_report_metadata_v1;
74+
75+
/** Opt-in envelope, not an extension of the caller-allocated ABI-4 report.
76+
* report.total_trades remains the logical total; report.trades_len and
77+
* report.equity_curve_len describe resident arrays only. In SUMMARY_ONLY
78+
* and STREAMED those arrays are NULL/0. The payload's existing free rules
79+
* apply; the envelope itself is caller-owned. */
80+
typedef struct pf_report_envelope_v1 {
81+
pf_report_metadata_v1 metadata;
82+
pf_report_t report;
83+
} pf_report_envelope_v1;
84+
85+
PF_STATIC_ASSERT(sizeof(pf_report_mode_v1_t) == 4);
86+
PF_STATIC_ASSERT(sizeof(pf_report_sequence_v1) == 16);
87+
PF_STATIC_ASSERT(offsetof(pf_report_sequence_v1, captured) == 8);
88+
PF_STATIC_ASSERT(offsetof(pf_report_sequence_v1, external) == 12);
89+
PF_STATIC_ASSERT(sizeof(pf_report_metadata_v1) == 128);
90+
PF_STATIC_ASSERT(offsetof(pf_report_metadata_v1, trades) == 16);
91+
PF_STATIC_ASSERT(offsetof(pf_report_metadata_v1, equity_curve) == 32);
92+
PF_STATIC_ASSERT(offsetof(pf_report_metadata_v1, broker_state_hash) == 48);
93+
PF_STATIC_ASSERT(offsetof(pf_report_metadata_v1, trace) == 64);
94+
PF_STATIC_ASSERT(offsetof(pf_report_metadata_v1, trace_names) == 80);
95+
PF_STATIC_ASSERT(offsetof(pf_report_metadata_v1, output_rows) == 96);
96+
PF_STATIC_ASSERT(offsetof(pf_report_metadata_v1, output_events) == 112);
97+
PF_STATIC_ASSERT(offsetof(pf_report_envelope_v1, report) == 128);
98+
PF_STATIC_ASSERT(sizeof(pf_report_envelope_v1) == 128 + sizeof(pf_report_t));
99+
100+
#ifdef __cplusplus
101+
} /* extern "C" */
102+
#endif
103+
104+
#endif /* PINEFORGE_REPORT_MODE_H */

‎scripts/broker_state_hash_waivers.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ margin_call_enabled_ # Source provider configuration (PineStrategyHost since L12
2828
path_order_mode_ # Requested path-mode configuration; selected native path and current decision context are hashed by the consumer.
2929
probe_suppress_tail_logic_ # Source provider configuration (PineStrategyHost since N14, formerly BacktestEngine); reporting/probe mode that does not alter accepted historical execution decisions.
3030
qty_step_ # Symbol metadata carrier; quantized request/term facts preserve the executable value at placement.
31-
range_end_trades_ # Report-only synthetic rows, never a live matching population or future admission input.
31+
range_end_trades_ # Report-only terminal mark descriptor (O1W R12), never a live matching population or future admission input; rows are rebuilt for the report only.
3232
realtime_tail_ # Source provider run-mode configuration (PineStrategyHost since N14, formerly BacktestEngine) consumed at begin; native stream lifecycle/hash owns active continuation state.
3333
realtime_tail_horizon_bars_ # Source provider run-mode configuration (PineStrategyHost since N14, formerly BacktestEngine) consumed at begin; it is not mutated by or read from broker settlement.
3434
security_calling_close_ms_ # Source-host auxiliary-slice publication cursor, zero outside the slice feed; no order matching or account settlement branch consumes it.

0 commit comments

Comments
 (0)