Skip to content

Commit 8e558eb

Browse files
luisleo526claude
andcommitted
Say what the per-bar broker hash is for, where its API is documented (R5 gap lane Q4)
The ruling the preceding commit pinned, stated at the three places the audit named plus the one sentence elsewhere that contradicted it. Documentation and comments only; no code, no behaviour, no epoch decision. * include/pineforge/engine.hpp — set_broker_state_hash_recording's comment now says what a row is (the run's continuation identity at that bar, not its trade outcome), that NativeRunPhase is folded into it on purpose, the two directions of the contract, the measured fact that the broker half alone is driving-mode invariant, and where the batch<->stream oracle actually lives. * include/pineforge/pineforge.h — the same contract in the C doxygen, in the terms a C caller has: the len identity is what survives across drivings, the rows are not comparable between a run() and a stream_begin, and what pins batch<->stream is the outcome itself. * docs/pages/native-engine.md — the paragraph's "A row is a function of the run's past only ... which is what makes the array a replay check" replaced by the qualified statement and a bullet each for same-driving, different- driving and the invariant broker half; the oracle named by artefact (tests/test_native_margin_fx_roll.cpp section 8, tests/test_streaming.cpp, scripts/check_corpus_parity.sh for the adapter). * include/pineforge/native_c_api.h — "Two runs that folded the same declarations and the same inputs answer the same value" was false across drivings and is now qualified to "driven the same way", with the batch-vs- stream case spelled out. Minimal and local: lane P4 owns this file. Anchors. The paragraph's own two citations were stale and are corrected: engine.hpp:2867 (past EOF, the header is 2803 lines) -> engine.hpp:2781, and native_execution_consumer.cpp:6519 -> :6630, both re-read on this tree. The engine.hpp and pineforge.h insertions shift the lines below them, so the three citations that pointed there are re-anchored too (pineforge.h:1029 -> :1043, :1037 -> :1051, :1063 -> :1077). Pre-existing drift elsewhere — engine.hpp:2897 and :2987 are past EOF, engine.hpp:2803 resolves to a closing brace — is untouched: it was already wrong and belongs to the documentation lane. Guards on this tree: check_c_abi_runtime, check_native_c_api_surface, check_native_cpp_versions, check_broker_state_hash_coverage and check_adapter_spec_shadowing all exit 0. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> (cherry picked from commit 50dd7fb209040322a12a5db8aefa00845135037f)
1 parent 4ddbcb2 commit 8e558eb

6 files changed

Lines changed: 89 additions & 10 deletions

File tree

‎docs/design/native-feature-parity.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -212,7 +212,7 @@ Columns: **Feature** (adapter mechanism, cited) · **Native today** (`yes` / `pa
212212
| i | `pf_pending_order_v1_t` Pine-named fields | pending_order_mirror.hpp:73, pending_order_mirror.hpp:135, pending_order_mirror.hpp:147-156, pending_order_mirror.hpp:164-169; enumerated by pending_order_mirror.cpp:140-174; written only by pine_adapter.cpp:16317-16346 | no change | Freeze (ABI). The neutral view arrives beside it (L7 / L13) | O (single-source) |
213213
| j | Pine-shaped lot flags | `PyramidEntry::market_pyramid_add` engine.hpp:247 (written only pine_strategy_host.cpp:500, read pine_adapter.cpp:15533, hashed engine_state_hash.cpp:57); `ordinary_market_open`, `pooc_terminal_market_entry`, `ordinary_stop_open`, `bracket_slot_shadowed` engine.hpp:242-283 have no writer in `src/` or `include/` yet are hashed (F); `entry_path_position` engine.hpp:253 is never written, hashed engine_state_hash.cpp:58 (O) | HASH-VISIBLE | Move the live flag to an adapter side table keyed by `entry_incarnation` (the adapter already keeps `PlacementTable` pine_adapter.hpp:297-476) and fold it through `hash_source_extension`; delete the dead ones; update the hash-coverage waivers. Whether a kernel TU beyond the hash reads them is unverified (O) | F O S |
214214
| k | `margin_call_enabled_` | engine.hpp:606, setter engine.hpp:3294-3298; no kernel reader; forwarded to the adapter pine_strategy_host.cpp:306 | HASH-VISIBLE (R5-8) | After L4 the kernel toggle is the presence of `spec.margin`. Move the flag and its C setter semantics into the source host; the C symbol stays (pinned runtime list) | F O |
215-
| l | Kernel decodes adapter strings | `closed_trade_close_cause` engine_trade_accessors.cpp:155-167 branches on `"__margin_call__"` and two `"Close Position (…)"` prefixes that only `src/source/**` writes; the stale comment engine_trade_accessors.cpp:145-149 cites the deleted `engine_risk.cpp`; it implements a public C contract pineforge.h:1037, pineforge.h:1063-1065 | HASH-VISIBLE (adds durable state) | Kernel `enum class CloseCause` recorded on the `Trade` at settlement (the kernel sets `MarginCall` when L4 liquidates; a source virtual covers adapter causes). C numbers unchanged. After L4 | O (single-source) |
215+
| l | Kernel decodes adapter strings | `closed_trade_close_cause` engine_trade_accessors.cpp:155-167 branches on `"__margin_call__"` and two `"Close Position (…)"` prefixes that only `src/source/**` writes; the stale comment engine_trade_accessors.cpp:145-149 cites the deleted `engine_risk.cpp`; it implements a public C contract pineforge.h:1051, pineforge.h:1077-1065 | HASH-VISIBLE (adds durable state) | Kernel `enum class CloseCause` recorded on the `Trade` at settlement (the kernel sets `MarginCall` when L4 liquidates; a source virtual covers adapter causes). C numbers unchanged. After L4 | O (single-source) |
216216
| m | `request.security` machinery | engine_security.cpp:22, engine_aux_security.cpp:89-140, `SecurityEvalState` engine.hpp:1974-2178 carry Pine semantics (lookahead, gaps, Heikin-Ashi, KI-55 range start) and are driven only by the source layer | HASH-VISIBLE / measured path | F planned to move both TUs and the state into the source host after native HTF. **R5-6 narrows this:** L6 reuses the feed store and routing, so those stay kernel; only the Pine-only evaluator semantics remain candidates. Do last. **Status (lane L12b): done** — the evaluator semantics are `source::PineSecurityEvalState` + `src/source/pine_security_eval.cpp`; the kernel keeps the registry, aggregator, feed store and one generic step | F S |
217217
| n | S-only proposals, not adopted | A generic `FillModel` interface with a TV model installed by the adapter (for the TV money / tick / path comments and helpers engine.hpp:1125-1348, engine.hpp:1514-1621); "emit generic FIFO facts and let an adapter policy produce the KI-62 scratch" | — | Superseded by R5-3 (price-grid lane) and R5-1 (TV fill specifics stay behind `resolve_execution_terms`); the dead-helper part is item e; there is no scratch code left to re-lower (item d) | S |
218218
| o | `pineforge::admission` (market-admission journal) — **N14 addition** | `include/pineforge/market_admission.hpp` (275 lines, installed by a kernel-only build) + `src/market_admission.cpp` (kernel TU): the observation journal of TradingView admission reviews; every `Configuration` field is a `strategy()` declaration parameter. No kernel TU consumed it (engine.hpp included it and used nothing; the hash helper in `src/broker_state_hash_internal.hpp` had one caller, `src/source/pine_state_hash.cpp`) | NEUTRAL (pure move; the fold is unchanged, so no hash domain moves) | **Status (N14): done** — moved whole to `include/pineforge/source/market_admission.hpp` + `src/source/market_admission.cpp` (`PINEFORGE_SOURCE_LAYER_SOURCES`); `hash_admission_field` moved verbatim into the source hash TU; `check_market_admission_schema.py` re-pointed; `MarketAdmissionDraft` / `MarketAdmissionJournal` spellings and the `market_admission_v2` inline namespace unchanged. ADR-0001 no longer calls it a generic stem | audit (Opus N14) |

‎docs/pages/native-engine.md‎

Lines changed: 42 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1394,19 +1394,56 @@ real series.
13941394
The per-bar **broker-state hash** is a row of that same report, so
13951395
`KernelRecorded` records it too. It stays behind the recording switch it
13961396
always had — `set_broker_state_hash_recording(true)`
1397-
(`engine.hpp:2867`; C: `strategy_set_broker_state_hash_recording`), off by
1397+
(`engine.hpp:2781`; C: `strategy_set_broker_state_hash_recording`), off by
13981398
default, set while no run is active — because each row is a full
13991399
`broker_state_hash()` over the lots and the closed rows. With the switch on,
14001400
one row follows each point, after the extremes that point just folded
1401-
(`record_script_report_point`, `native_execution_consumer.cpp:6519`), so
1401+
(`record_script_report_point`, `native_execution_consumer.cpp:6630`), so
14021402

14031403
```text
14041404
broker_state_hash_len == equity_curve_len == script_bars_processed
14051405
```
14061406

1407-
in batch and across a stream's warmup and realtime legs alike. A row is a
1408-
function of the run's past only: the row after bar *k* equals the last row of
1409-
a run that ended at bar *k*, which is what makes the array a replay check.
1407+
in batch and across a stream's warmup and realtime legs alike. That length
1408+
identity is the only part of the array that holds across drivings.
1409+
1410+
**What a row is, and what it is for.** A row is the run's *continuation
1411+
identity* at that bar, not its trade outcome. `broker_state_hash()` is
1412+
`broker_state_hash_from_execution_hash(continuation_hash())`, so a row folds
1413+
the kernel's broker state — the lots, the realized sums, the equity extremes,
1414+
the closed rows — and, ahead of it, the state a resume would continue from.
1415+
`NativeRunPhase` (`Batch` / `Warmup` / `Realtime`, readable as
1416+
`native_state().phase`) is folded into that continuation **on purpose**: a
1417+
consumer mid-warmup and a consumer mid-realtime are not interchangeable
1418+
continuations, which is exactly what `native_continuation_hash()` is for and
1419+
why `stream_state_hash()` leads with this value. So the array is a replay
1420+
check **within one driving mode** and deliberately not across modes:
1421+
1422+
- *Same driving.* Two runs driven the same way over the same bars record the
1423+
same rows, and the row after bar *k* equals the last row of a run driven the
1424+
same way that ended at bar *k*. That prefix closure — in a batch, and in a
1425+
stream at any warmup split — is what makes the array a replay check.
1426+
- *Different driving.* The same bars booking the same trades record
1427+
**different** rows under `run()`, under `stream_begin(warmup=1)` +
1428+
`stream_push_bar` and under `stream_begin(warmup=all)`, differing from index
1429+
0. Two streams share exactly the bars both are still in `Warmup` for, and
1430+
converge again one bar past the later warmup boundary, when their realtime
1431+
legs become the same continuation.
1432+
- *The broker half alone is driving-mode invariant.* Factor the continuation
1433+
out — `broker_state_hash_from_execution_hash(fixed)`, protected on
1434+
`BacktestEngine` — and the remaining fold is identical at every bar in every
1435+
driving. The divergence is the continuation and nothing else.
1436+
1437+
**The batch↔stream oracle is the outcome, not the hash.** What pins that a
1438+
stream books what a batch books is the outcome twin: section 8 of
1439+
`tests/test_native_margin_fx_roll.cpp` compares every offered check point,
1440+
every measured requirement, every receipt, every closed row and the book that
1441+
is left; `tests/test_streaming.cpp` compares trade count, position, equity and
1442+
each closed row's identity, prices and size. For the Pine adapter the same job
1443+
is done over the whole validation corpus by `scripts/check_corpus_parity.sh`.
1444+
Both directions of the row contract are pinned in
1445+
`tests/test_native_report_truth.cpp`.
1446+
14101447
Recording is reporting — the switch is not continuation state and no row is
14111448
ever read back, so a run records the same trades, curve, continuation and
14121449
broker state with it on or off. Under `HostRecorded` the report is the host's

‎docs/pages/pine-to-native.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ Everything here is one call: `submit(Request)` native_host.hpp:487, or
120120
| `time`, `bar_index`, session facts | `NativeDecisionContext` market_driver.hpp:84, `NativeCoordinate` market_driver.hpp:46 | Copied onto the callback stack; mutating it changes nothing. |
121121
| `strategy.position_size`, `strategy.position_avg_price` | `physical_position()` native_host.hpp:498 | |
122122
| `strategy.equity` | `native_marked_equity(mark)` native_host.hpp:499 | |
123-
| `strategy.closedtrades`, `strategy.closedtrades.*` | `closed_trade_count()` / `closed_trade(i)` engine.hpp:2395-2396, a `Trade` row; C: `pf_report_t::trades` (`pf_trade_t`) plus `strategy_closed_trade_entry_id` / `_exit_id` / `_exit_comment` / `_close_cause` / `_entry_incarnation` pineforge.h:1029-1089 | Field by field in [Open and closed trades](@ref pine_to_native_map_trades). `report_trade_count()` / `get_report_trade(i)` engine.hpp:2403-2406 span the same rows followed by the range-end rows. |
123+
| `strategy.closedtrades`, `strategy.closedtrades.*` | `closed_trade_count()` / `closed_trade(i)` engine.hpp:2395-2396, a `Trade` row; C: `pf_report_t::trades` (`pf_trade_t`) plus `strategy_closed_trade_entry_id` / `_exit_id` / `_exit_comment` / `_close_cause` / `_entry_incarnation` pineforge.h:1043-1089 | Field by field in [Open and closed trades](@ref pine_to_native_map_trades). `report_trade_count()` / `get_report_trade(i)` engine.hpp:2403-2406 span the same rows followed by the range-end rows. |
124124
| `strategy.netprofit`, equity curve, drawdown | partial — `fill_report` engine.hpp:3161 | The trade list and trade statistics are filled; for a bare native host the **equity curve is empty** (the recorders are protected, engine.hpp:2386-2412) so equity metrics degenerate silently engine_metrics.cpp:168, and a position still open at the end has no range-end row engine_run.cpp:191. **Lane L2**. |
125125
| `strategy.opentrades`, `strategy.opentrades.*` | `native_open_lots(mark)` native_host.hpp:861 → one `NativeOpenLot` native_host.hpp:255 per open physical lot; C: `strategy_native_open_lot_count_v1(s, mark)` / `strategy_native_open_lot_get_v1(s, i, &row)` native_c_api.h:871-879 → `pf_native_open_lot_v1` native_c_api.h:509 | Owning snapshot of the book lot by lot, oldest first, marked at the price you pass (Pine marks at the current `close`; pass the bar's close in `on_native_bar`, the fill price in `on_native_applied`, the print in `on_native_tick`). Observation only: it moves no fill, no hash and no row. Field by field in [Open and closed trades](@ref pine_to_native_map_trades). |
126126

‎include/pineforge/engine.hpp‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2754,6 +2754,30 @@ class BacktestEngine {
27542754
// value; the flag itself is persistent configuration, like
27552755
// set_realtime_tail, so it must be set BEFORE stream_begin to also
27562756
// cover the warmup bars.
2757+
//
2758+
// What a row is, and what it is for. A row is the run's CONTINUATION
2759+
// IDENTITY at that bar, not its trade outcome: broker_state_hash() is
2760+
// broker_state_hash_from_execution_hash(continuation_hash()), so it folds
2761+
// the kernel's broker state AND, ahead of it, the state a resume would
2762+
// continue from. NativeRunPhase (Batch/Warmup/Realtime, readable as
2763+
// native_state().phase) is part of that continuation on purpose, because a
2764+
// consumer mid-warmup and one mid-realtime are not interchangeable
2765+
// continuations. So the array is a replay check WITHIN one driving mode
2766+
// and deliberately not across modes:
2767+
// * same driving -- two runs driven the same way record the same rows,
2768+
// and the row after bar k is the last row of a run driven the same way
2769+
// that ended at bar k (a batch and a stream at any warmup split);
2770+
// * different driving -- run(), stream_begin(warmup=1)+push and
2771+
// stream_begin(warmup=all) over the same bars booking the same trades
2772+
// record different rows from index 0; only the length identity above
2773+
// survives. Two streams share exactly their common Warmup prefix.
2774+
// * the broker half alone IS driving-mode invariant: factor the
2775+
// continuation out with broker_state_hash_from_execution_hash(fixed)
2776+
// and the remaining fold is identical at every bar in every driving.
2777+
// The batch<->stream oracle is therefore the OUTCOME, not this array:
2778+
// tests/test_native_margin_fx_roll.cpp section 8 and tests/test_streaming.cpp
2779+
// for the kernel, scripts/check_corpus_parity.sh for the Pine adapter.
2780+
// Both directions are pinned in tests/test_native_report_truth.cpp.
27572781
void set_broker_state_hash_recording(bool on) {
27582782
guard_native_mutation("set_broker_state_hash_recording");
27592783
broker_state_hash_recording_ = on;

‎include/pineforge/native_c_api.h‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1598,8 +1598,12 @@ PF_API int strategy_native_liquidation_price_v1(pf_strategy_t s, double* out);
15981598
PF_API int strategy_native_risk_state_v1(pf_strategy_t s, pf_native_risk_state_v1* out);
15991599

16001600
/** The run's continuation identity — `native_continuation_hash()`. Two runs
1601-
* that folded the same declarations and the same inputs answer the same
1602-
* value; it is the C spelling of the hash a stream resumes against. */
1601+
* driven the same way that folded the same declarations and the same inputs
1602+
* answer the same value; a batch and a stream over identical bars booking
1603+
* identical trades do NOT, because the driving mode is part of a
1604+
* continuation. It is the C spelling of the hash a stream resumes against,
1605+
* and the same reason the per-bar broker-state hash is per driving mode
1606+
* (`pineforge.h`, `strategy_set_broker_state_hash_recording`). */
16031607
PF_API int strategy_native_continuation_hash_v1(pf_strategy_t s, uint64_t* out);
16041608

16051609
/** Open a cohort roster for PF_NATIVE_OWNER_BIND_COHORT. */

‎include/pineforge/pineforge.h‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -883,7 +883,21 @@ PF_API int strategy_last_bar_dual_entry_path(pf_strategy_t s);
883883
* returns misses the warmup bars already dispatched.
884884
* Default off (@p on == 0): pf_report_t::broker_state_hash is NULL /
885885
* 0-length and every historical run stays byte-identical to before this
886-
* flag existed. */
886+
* flag existed.
887+
*
888+
* What a row is, and what it is for. A row is the run's CONTINUATION
889+
* IDENTITY at that bar, not its trade outcome: it folds the broker state and,
890+
* ahead of it, the state a resume would continue from -- the driving mode
891+
* (batch run, stream warmup, stream realtime) included, on purpose. So the
892+
* array is a replay check WITHIN one driving mode and deliberately not across
893+
* modes: two runs driven the same way over the same bars record the same
894+
* rows, and a run driven the same way that ended at bar k recorded, as its
895+
* last row, the row the longer run recorded after bar k; but a run(), a
896+
* #strategy_stream_begin with one warmup bar and one with every bar as warmup
897+
* record DIFFERENT rows from index 0 over the same bars booking the same
898+
* trades. Only the len == script_bars_processed identity above holds across
899+
* drivings. What pins that a stream books what a batch books is the outcome
900+
* itself -- the trades, the position and the equity -- and not this array. */
887901
PF_API void strategy_set_broker_state_hash_recording(pf_strategy_t s, int on);
888902
/** Return the broker-state hash of the FINAL state after the most recent
889903
* run() (see #strategy_set_broker_state_hash_recording's doc and

0 commit comments

Comments
 (0)