Skip to content

Commit f48b7c8

Browse files
luisleo526claude
andauthored
Deliver a native run's report only for the call that began it (#361)
* Deliver a native run's report only for the call that began it (onto #356) One integration commit on main 70ddaa3 (recorded outputs, #356): the net change 31a40a1..d83ab7d4a of px/native-attempt-fix-20261009, 15 files, applied as a squash of that range. The author branch and its history are untouched. Supersedes 3d2f3f4 (the per-entry Completed gate): the reader gate 86a1856, its revert 124a434 and the restore ec9047d are folded into this net diff, so the documentation 3d2f3f4 wrote is carried here in its final form and this commit removes no documentation sentence of main. What changes (unchanged from the author head): - BacktestEngine::fill_report publishes the retained rows only while the latest run or stream_begin call began (reached reset_run_state). A call refused without beginning (a begin outside Ready, a refused bar array or run option, a calendar or timezone refusal, a begin over a live stream) gets the empty report, and strategy_native_run_v1 answers PF_NATIVE_E_RUN_FAILED for it instead of handing out the earlier run's report. - NativeExecutionConsumer keeps two bools, rows_current_ and batch_current_, behind a private RAII AttemptScope opened first by run_simple, run_tf, run_rich and stream_begin. strategy_stream_fill_report goes through native_stream_snapshot_report and fill_snapshot_report, so a live stream's rows survive a batch call refused over it, which reads empty. - No engine.hpp, ReportC or public API layout change, no codegen change, no version bump. The two bools are report-presentation bookkeeping and are not hashed; identity of the state hashes is owed on the parity population. - Tests kept: the 14 repro assertions (RED on 31a40a1) and the old, twin, current-partial and live-stream rows in tests/test_native_c_api.c, test_checked_settings.cpp and test_native_example_batch.cpp; one obsolete row in test_run_failure_codes.cpp (a second call on a Failed handle is refused, 0 trades). Integration against #356: - One textual conflict, docs/design/native-feature-parity.md rows OT6 and OT7 (adjacent lines changed by different sides). Kept #356's OT6 ("73 runtime exports") and the author's OT7 anchor (native_execution_consumer.cpp:7636). Nothing from either side was dropped. - The other five files #356 also changed (ADR 0001, native-engine.md, native_c_api.h, c_abi.cpp, engine_report.cpp) merged cleanly and keep every recorded-output implementation and doc line. - Six documentation anchors that the author's insertions moved, on lines the author's diff left alone, are carried by an exact line map: parity page lines 126, 159 and 1679, native-engine.md line 647, pine-to-native.md lines 129 and 466. - Not built, not run and not anchor-checked here (no compiler or project helper on this lane); the verification runbook is in the lane report. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * Close the report-ownership review findings: pin, nested and snapshot controls, guide wording Follow-up to 44f7d0f (Deliver a native run's report only for the call that began it, onto #356), which integrated the 3d2f3f4 .. d83ab7d4a range onto main 70ddaa3. This commit answers the final review of that work (one P1, two P2) and changes no source under src/ or include/, no PF_API export and no admission behavior. P1, documentation content pin. The stream-report call in strategy_stream_fill_report (src/c_abi.cpp:770) sits inside the window c_abi.cpp:308-963 that docs/design/native-feature-parity.md row 8 pins by content, and a line-neutral edit does not preserve a content hash. The row's claim was re-read against the integrated source first: 73 PF_API definitions in c_abi.cpp, the 12-symbol stream family (strategy_stream_begin .. strategy_stream_fill_report), no export added or removed. The pin is refreshed from sha256:d19d72e6... to sha256:70be189a... by hashing the window's lines joined with newlines. All 15 content pins of the published pages were recomputed by text; 15 of 15 now match. P2, controls. tests/test_native_c_api.c gains check_nested_run_report_ownership: a genuine C callback, after a closed trade, makes a second strategy_native_run_v1 on its own handle over a poisoned output. The nested call must answer PF_NATIVE_E_RUN_FAILED with the empty report, while the outer run ends Failed (Contract, read from the state, not assumed Running) and publishes its own partial trade. tests/test_checked_settings.cpp gains test_snapshot_restores_batch_gate: after a batch call refused over a live stream, a normal and a throwing (injected allocation failure) strategy_stream_fill_report are followed by BacktestEngine::fill_report with no run in between, which must still answer empty. Both files only gain lines: the 14 reproduction assertions, the old, twin, live and started-then-failed controls are untouched. P2, guide. docs/pages/native-engine.md "A run's report." no longer says every listed refusal leaves the lifecycle, the run identity and the retained rows as it found them. It states the cause-by-cause behavior that exists unchanged on the base: a begin on a Completed, Failed or unconfigured handle and a refused bar array or run option preserve them; a calendar or timezone refusal comes after the consumed run-number high-water moved and fails the lifecycle; a begin while Running latches Failed (Contract, Begin; Contract, Configure on a generated source host). It also states that the batch reader stays empty after a snapshot and that a nested run call reads empty while the outer run keeps its rows. This rewords sentences introduced by 44f7d0f (and before it by 3d2f3f4, 86a1856, ec9047d): the dropped sentence "leaves the lifecycle, the run identity and the retained rows as it found them" is replaced, not restored. Not built, not run, not guard-checked here (no compiler or project helper on this lane). Focused GREEN and the integrated-tree proof are owed on a spot box. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * Move the stream snapshot bridge into the engine epoch the version guard checks The measured final run (green1, guards/05-native-versions.log) failed the declared source guard scripts/check_native_cpp_versions.py on 1c89e87 while the same guard passed on the newmain base 70ddaa3: ValueError: pineforge out-of-line method must belong to engine_script_run_v19 Cause: native_stream_snapshot_report was defined after the closing brace of `inline namespace engine_script_run_v19` in src/native_execution_consumer.cpp. The guard compares every qualified `Name::member(` match in `namespace pineforge` with the matches inside the epoch, so the bridge's `NativeExecutionConsumer::fill_snapshot_report(` call counted as outside the epoch (guards-mid/diag.mid.txt: {'NativeExecutionConsumer::fill_snapshot_report': 1}); the next comparison, namespace_functions, would have named the bridge itself. Fix, two line-neutral hunks and nothing else: - src/native_execution_consumer.cpp: the six-line bridge (two comment lines, the three-line definition, one blank line) moves above the epoch's closing brace. The definition and the fill_snapshot_report, rows_are_current and AttemptScope bodies are byte-identical; one comment line gains ", in the engine epoch". The pinned native_run_spec_v3 digest block stays where it was. - src/c_abi.cpp: the forward declaration beside the two native_c_host hooks is reopened in `inline namespace engine_script_run_v19`, on its own single line (the shape market_driver.hpp uses for its native_run_spec_v3 forward declaration), so it names the same entity as the definition. The call pineforge::native_stream_snapshot_report( in strategy_stream_fill_report is unchanged: qualified lookup finds the member of the inline namespace. The mangled name gains the epoch component, and the two files above are the only mentions of the bridge in the tree. No PF_API export, header, layout, hash, checker, allowlist or version changes. Both files keep their line counts (c_abi.cpp 1173, native_execution_consumer.cpp 11557) and every line outside the touched hunks keeps its number, so no file:line anchor and none of the content pins of the published pages move. All 21 pin occurrences were re-hashed by text on this tree and match (the six c_abi.cpp windows start at line 308); 137 anchors into these two files were scanned and none touches a changed line. No documentation changes. Not compiled or run here (local-only lane). A read-only text replay of the guard's ownership comparison (stdlib only, the guard itself not executed) reproduces the measured difference exactly on 1c89e87; on this tree TYPE_DEF, ALIAS_DEF, METHOD_DEF and namespace_functions agree for the consumer .cpp and .hpp. The build, ctest, the declared source guards and the doc gates still have to be re-run on a spot box. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * Correct attempt-report citations and qualify refusal documentation Apply the retained docdelta1 patch after the private bridge epoch repair. This corrects 1c89e87 and the earlier 44f7d0f citation mapping. Replace the broad claim "A calendar or timezone refusal comes after the consumed run-number high-water moved" with its begin_ready scope. Replace "A begin while the handle is Running" with the actual batch/reentrant path and the repeated-stream exception. Four citations now point at the source they describe. Runtime behavior is unchanged. The author timed out; the patch and sidecar predate the bound, while its detailed report was written after it. Final independent review and exact-head runtime checks remain required. --------- Co-authored-by: Claude Sonnet 5.5 <[email protected]>
1 parent 70ddaa3 commit f48b7c8

15 files changed

Lines changed: 1521 additions & 127 deletions

‎docs/adr/0001-kernel-adapter-boundary.md‎

Lines changed: 21 additions & 21 deletions
Large diffs are not rendered by default.

‎docs/design/native-feature-parity.md‎

Lines changed: 59 additions & 59 deletions
Large diffs are not rendered by default.

‎docs/native-refactor-progress.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@ them for the record only. "Linux aarch64" is the maintainers' Cortex-A725 host.
173173

174174
| Item | What it is | Measured size | Deferred by | Prototype and evidence | Reopens when |
175175
| --- | --- | --- | --- | --- | --- |
176-
| D2-B, the quiet-bar protocol and one receipt read per bar | PERF-D2's lane: a per-bar bar-open request, a bulk fold of quiet driver points, the adapter's quiet-open predicate with lazy open bookkeeping, and a gate before the receipt poll. D2-A landed the hook declarations and the O(1) `has_command_after` (`src/native_execution_consumer.hpp:286`). PERF-L4 gated the read inside `observe_terminal_receipts` (`src/source/pine_adapter.cpp:6307`), which still runs about three times a bar | PERF-D2, Linux aarch64 (Cortex-A725), callgrind and GBench: about ×0.88 on the 14 probes (an estimate from a measured −13 to −14 % floor bound for the bar-open request alone); after PB the receipt poll still costs about 250 instructions a bar on the no-order floor. On this tree the quiet-bar probe asks the receipt gate 40,996 times and each once-per-bar-open gate 13,248 times: 3.09 reads a bar, 82 % of them stopped at the gate (macOS arm64, AppleClang 17, Release) | "R5 PERF-D2 DONE 2026-09-24 19:56" (D2-B after V19-D); "R5 SUPERVISOR RULING 2026-09-25 00:15 Taipei": "D2-B (quiet-bar protocol) waits for PERF-D3's verdict"; TRIAGE1 §D | An unpublished prototype branch and its patches. PERF-D3's S1 (next row) prototyped the request and the lazy opening | A post-release performance epoch. Done when the receipt gate is asked as often as the once-per-bar gates |
176+
| D2-B, the quiet-bar protocol and one receipt read per bar | PERF-D2's lane: a per-bar bar-open request, a bulk fold of quiet driver points, the adapter's quiet-open predicate with lazy open bookkeeping, and a gate before the receipt poll. D2-A landed the hook declarations and the O(1) `has_command_after` (`src/native_execution_consumer.hpp:295`). PERF-L4 gated the read inside `observe_terminal_receipts` (`src/source/pine_adapter.cpp:6307`), which still runs about three times a bar | PERF-D2, Linux aarch64 (Cortex-A725), callgrind and GBench: about ×0.88 on the 14 probes (an estimate from a measured −13 to −14 % floor bound for the bar-open request alone); after PB the receipt poll still costs about 250 instructions a bar on the no-order floor. On this tree the quiet-bar probe asks the receipt gate 40,996 times and each once-per-bar-open gate 13,248 times: 3.09 reads a bar, 82 % of them stopped at the gate (macOS arm64, AppleClang 17, Release) | "R5 PERF-D2 DONE 2026-09-24 19:56" (D2-B after V19-D); "R5 SUPERVISOR RULING 2026-09-25 00:15 Taipei": "D2-B (quiet-bar protocol) waits for PERF-D3's verdict"; TRIAGE1 §D | An unpublished prototype branch and its patches. PERF-D3's S1 (next row) prototyped the request and the lazy opening | A post-release performance epoch. Done when the receipt gate is asked as often as the once-per-bar gates |
177177
| S1a, kernel quiet-bar driving (PERF-D3) | A bar with nothing live, a flat book and a declined opening is driven in a quiet form: the kernel writes the bookkeeping its four price points would leave (ordinals, the last two driver marks, the floor, the point epochs) without walking them or calling the bar-open callback. Additive host cadence declarations; C hosts declare from their callback table; one C symbol under the PF_API checklist | PERF-D3, Linux aarch64 (Cortex-A725), GBench time and `perf stat` instructions, on the L3 tree `b33ea659`, S1a with S1b: no-order floor 0.764 / 0.660 in time (0.718 / 0.656 in instructions), 14 probes 0.958 / 0.921 (0.892 / 0.856), 100 slots 0.972 / 0.956 per slot. A bare kernel host that opts in: idle 0.545× (35 % fewer instructions a bar), market 0.81×, bracket 0.92×. Byte-identical: corpus 312/312, the 132-configuration battery with its hash columns | "FINDING PERF-D3 (filed for a post-release epoch per the owner's cap)" (ledger, 2026-09-24 16:58 UTC); "OWNER DECISION 2026-09-25 00:20 Taipei"; TRIAGE1 §D: "PERF-D3 S1/S2 (post-release epoch)" | An unpublished prototype branch (five commits on L3's tree) and its patches | The post-release epoch, with the owner calls PERF-D3 lists (the API under the existing ruling, the restated gate count) |
178178
| S1b, the Pine lazy opening | The Pine host records what its skipped opening and its uncalled input callback would have written, at the bar's first host callback. Member functions only, no layout change | Measured with S1a. A quiet bar then asks 15 gates, not 30, so the quiet-bar cost check `CHECK(asked >= 20 * o.boundary.size())` (`tests/test_adapter_quiet_bar.cpp:1054`) must be restated | As S1a | As S1a. The in-script hash witnesses caught the prototype's one miss (`awaiting_legacy_script_open_ms_`) | As S1a |
179179
| S1c, the held-position wake band | S1 for bars that hold a position with nothing live, 41 % of the median public slot's bars. The host declares a price band, and the kernel calls the opening only when the bar's range leaves it. The adapter derives the band from its margin model and declines when it cannot bound it | PERF-D3, a measurement-only upper bound without the band, which is not sound in general: 100 slots 0.776 / 0.716 per slot, 14 probes 0.830 / 0.761, cumulative against L3 with S1, S2 and D2-A | As S1a; the owner call "Fund the price band" | An unpublished measurement-only script | As S1a. Size L, parity risk high: TradingView's margin money rules |
@@ -184,7 +184,7 @@ them for the record only. "Linux aarch64" is the maintainers' Cortex-A725 host.
184184
| A checked-values handoff for executions (L3) | `execution_values` runs twice, in `check_execution` (`src/native_order.cpp:4791`) and again in `apply_execution` (`src/native_order.cpp:4805`); handing the checked values across needs a public type, an API addition with no layout change | L3, as above: 2.21 % of bracket for the two together, about 1.1 % recoverable | L3 ruling 2: "a public checked-values handoff for executions (~1.1% bracket) -> DEFERRED" | As above | An owner decision on the API addition |
185185
| A chunked `trades_` store (L3) | Closed trades in chunks rather than one vector; `trades_` (`include/pineforge/engine.hpp:546`) is protected `BacktestEngine` state, so script ABI | L3, as above: on bracket, vector growth 0.64 %, `Trade` moves 0.74 % and `record_close_trade` 0.43 % (at most about 1.4 %); at most 0.35 % on the 14 probes | L3 ruling 4: "chunked trades_ store (ABI) -> DEFERRED" | As above | An `engine_script_run` epoch that changes the engine's protected layout |
186186
| A zoned chart-day memo (D2-D finding 4) | On a chart with a timezone, `libc_chart_day_key` (`src/source/pine_adapter.cpp:17130`) takes the process timezone lock through `ScopedTimezone` and calls `localtime_r` on every bar, because `on_bar_open` (`src/source/pine_adapter.cpp:22011`) reads `chart_day_key` (`src/source/pine_adapter.cpp:22248`) unconditionally. The non-UTC monthly Sharpe/Sortino walk, `month_key_local` (`src/engine_metrics.cpp:33`), pays one `localtime_r` per equity point. UTC charts use D2-D's memo, `civil_chart_day_key` (`src/source/pine_adapter.cpp:17114`) | Not measured on a zoned chart. The UTC path cost 134 instructions a bar before D2-D's memo and 68 after (D2-D, Linux aarch64, callgrind on the floor). PERF-K1: `ScopedTimezone` copies the zone name, which allocates for names of 16 or more characters under libstdc++ | TRIAGE1 §D: "D2-D finding 4 (the timezone `localtime_r` memo)", under the owner's cap. D2-D's report: an exact zoned memo "would need the zone's transition table, which belongs in `timezone.cpp`" | None; the UTC memo's witness is `tests/test_chart_day_memo.cpp` | A lane that owns `src/timezone.cpp` and builds an exact zone-transition memo (PERF-K1's Sitka case shows why a shortcut is not exact), byte-identical, with the zoned keys of `test_chart_day_memo` still equal to the computed ones |
187-
| Thread-local reads that remain per call | D2-C moved the three scopes' writes onto the pump's block, `pump_ambient` (`src/native_execution_consumer.hpp:461`). A reader still reaches the block through `tl_runtime_ambient` (`src/ta_extremes_volume.cpp:29`): `bar_context` (`src/ta_extremes_volume.cpp:82`), read by every `ExtremeRing::update` (`src/ta_extremes_volume.cpp:117`), which backs `ta.highest`, `ta.lowest`, `ta.highestbars` and `ta.lowestbars` and, through an embedded `Highest` and `Lowest`, `ta.stoch`, `ta.wpr` and `ta.range`; and `matching_day_partition` (`src/timeframe.cpp:628`), read by the default-anchored `ta.vwap` through `session_day_index`, by `crosses_boundary` on a DAY period (`timeframe.change("D")`) and by the session-period helpers. `decompose_ms_local` (`src/session_time.cpp:250`) reads two thread-locals per call on a zoned clock, and the prepared order storage's `free_blocks` (`src/native_order.cpp:605`) one per prepared mutation. `ema_na_warmup_flag` is read once per EMA, at its first compute | One thread-local access per reader call. AUDIT4, macOS arm64, a dlopen'd tutorial-MACD probe, counted by `_tlv_get_addr` breakpoints and by descriptor patching: 1.000 a bar for one `ta::Highest` and for one default `ta::VWAP`, 1.999 for one `pine_hour` on a New York clock, 0.000 on the no-order floor and with orders. On Linux a TLS-descriptor call costs about 20 instructions: D2-C's 13 → 4 calls a bar was 260 → 80 instructions (Linux aarch64, GCC 13, callgrind). INT23: 2.00 a bar on the zoned probe 064, 0.08 on probe 008 | None for the TA and day-partition readers: "RULING: INT23 ACCEPTED" (ledger, 2026-09-24 23:30 UTC) reads "dlopen TLS 13->0/bar", and INT23's report names only the timezone and order-core residuals. This row is their record | Unpublished (AUDIT4's scratch) | A byte-identical lane that hands these readers the pump's block without a thread-local read. `ta::bar_context()` is script ABI, so its public declaration stays as it is |
187+
| Thread-local reads that remain per call | D2-C moved the three scopes' writes onto the pump's block, `pump_ambient` (`src/native_execution_consumer.hpp:470`). A reader still reaches the block through `tl_runtime_ambient` (`src/ta_extremes_volume.cpp:29`): `bar_context` (`src/ta_extremes_volume.cpp:82`), read by every `ExtremeRing::update` (`src/ta_extremes_volume.cpp:117`), which backs `ta.highest`, `ta.lowest`, `ta.highestbars` and `ta.lowestbars` and, through an embedded `Highest` and `Lowest`, `ta.stoch`, `ta.wpr` and `ta.range`; and `matching_day_partition` (`src/timeframe.cpp:628`), read by the default-anchored `ta.vwap` through `session_day_index`, by `crosses_boundary` on a DAY period (`timeframe.change("D")`) and by the session-period helpers. `decompose_ms_local` (`src/session_time.cpp:250`) reads two thread-locals per call on a zoned clock, and the prepared order storage's `free_blocks` (`src/native_order.cpp:605`) one per prepared mutation. `ema_na_warmup_flag` is read once per EMA, at its first compute | One thread-local access per reader call. AUDIT4, macOS arm64, a dlopen'd tutorial-MACD probe, counted by `_tlv_get_addr` breakpoints and by descriptor patching: 1.000 a bar for one `ta::Highest` and for one default `ta::VWAP`, 1.999 for one `pine_hour` on a New York clock, 0.000 on the no-order floor and with orders. On Linux a TLS-descriptor call costs about 20 instructions: D2-C's 13 → 4 calls a bar was 260 → 80 instructions (Linux aarch64, GCC 13, callgrind). INT23: 2.00 a bar on the zoned probe 064, 0.08 on probe 008 | None for the TA and day-partition readers: "RULING: INT23 ACCEPTED" (ledger, 2026-09-24 23:30 UTC) reads "dlopen TLS 13->0/bar", and INT23's report names only the timezone and order-core residuals. This row is their record | Unpublished (AUDIT4's scratch) | A byte-identical lane that hands these readers the pump's block without a thread-local read. `ta::bar_context()` is script ABI, so its public declaration stays as it is |
188188
| "audit-off mode", "LTO", "data-oriented layout" | Named in "OWNER DECISION 2026-09-25 00:20 Taipei" among the post-release structural items: "S1 bulk quiet bars, S2 per-strategy specialization, audit-off mode, LTO+PGO, data-oriented layout". No engine or campaign document defines, designs or measures the three | None | The owner decision | None | Struck from this inventory until a design note defines one of them |
189189

190190
### Recorded witness drift
@@ -194,9 +194,9 @@ them for the record only. "Linux aarch64" is the maintainers' Cortex-A725 host.
194194
K4, the batch log pre-size. Since V19-B's step 6 (`08e71bae`) a run under the
195195
default `NativeEventRetention::Window` keeps no driver point and retires its
196196
journal at every script-bar boundary, so K4 has nothing to size.
197-
`reserve_driver_log` (`src/native_execution_consumer.cpp:2870-2875`) returns
197+
`reserve_driver_log` (`src/native_execution_consumer.cpp:2884-2889`) returns
198198
unless the retention is Full, and `presize_logs`
199-
(`src/native_execution_consumer.cpp:9655-9666`) sizes no journal under Window.
199+
(`src/native_execution_consumer.cpp:9669-9680`) sizes no journal under Window.
200200
`NativeRunSpec::event_retention` (`include/pineforge/native_run_spec.hpp:714`)
201201
defaults to Window, and the Pine adapter declares
202202
`NativeEventRetention::Window` (`src/source/pine_adapter.cpp:2931`). K4 still

0 commit comments

Comments
 (0)