Skip to content

Commit 4dc496d

Browse files
luisleo526claude
andauthored
Report timeframe fields restored; aggregated-chart admission no longer aborts (FIX-E1E2) (#299)
* FIX-E1E2 (E1): the report's script_tf_ratio and needs_aggregation again pf_report_t::script_tf_ratio and needs_aggregation read 0 on every run since #254 (73817c1) retired the legacy scheduler, whose run_tf_impl set both (pine_scheduler.cpp:1334-1335) and whose simple run set 1 / false. The native scheduler computes the same ratio at run begin (PineScheduler::run_begin) and now hands it to the report fields there, for every run the Pine host begins: the bare run, the timeframe-aware run, an auto-detected input, the magnifier, a stream. Report-only: no trade moves. test_report_timeframe_ratio runs each route and a reused host; at 35db01c it fails 19 of its 79 checks (every ratio and aggregation flag reads 0). ADR-0001 gains the row the kernel-seam-rows gate asks for (the timeframe report facts, report-only, outside the hashed region); doc anchors into pine_scheduler_native.cpp carried through the line map from 35db01c. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * FIX-E1E2 (E2): an aggregated chart's admission receipts name their command's bar A run with Pine v6's default sizing (100 % of equity), a commission and process_orders_on_close stopped on an aggregated chart with "admission receipt requires its exact earlier command" (market_admission.cpp:26). The adapter binds each entry command's admission observation to the row's projection bar, the input slot (projection_bar_index). Since #289 (0d76a09) the kernel's public interval_index is the script bar, and the receipts that follow a command kept reading it: the sizing revision a margin call makes of a pending default-quantity entry (refresh_pending_sizing_after_margin) and the three market reviews (DefaultGross, ExplicitPair, TerminalGross). On an aggregated chart the script bar is smaller than the input slot, so the receipt preceded its own command and Draft::sizing_revised / reviewed threw. The terminal review of a process_orders_on_close pair of fixed-quantity calls at pyramiding 0 hit the same check with no commission or percent sizing at all. Every receipt now carries projection_bar_index(context); on a chart run the two indices are equal. With the pair's terminal review running on an aggregated chart, its same-bar round trips were dated at the next bar's open: the reversing call closes the first leg's lot before the first leg's own notification re-dates it to the chart bar's open (PineStrategyHost::on_native_applied). The opening notification now also re-dates the rows a later execution of the same pass booked from its lot. TradingView's tapes of two synthetic probes written for this lane (tests/fixtures/aggregated_admission_receipt, BINANCE:ETHUSDT.P 60, lab tv --no-note): test_aggregated_admission_receipt_tapes replays both on the chart's bars and aggregated from 15 and from 5 minutes. At 35db01c the four aggregated rows stop with the receipt error (8 of 66 checks fail); with the receipts alone the pair's aggregated rows misdate eight round trips (16 of 150 fail); all 150 pass. ADR-0001's row of the kernel-row fields the Pine host amends names Trade::entry_time. Floors: RELEASE_MIN_TESTS 792 and EXCLUDED_REGISTERED_MIN 773/773/782 for the lane's two rows; doc anchors into pine_adapter.cpp and pine_strategy_host.cpp carried through the line map from 35db01c, and the RELEASE_MIN_TESTS citations to its new line. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
1 parent 35db01c commit 4dc496d

23 files changed

Lines changed: 1777 additions & 64 deletions

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,7 @@ What each gate refuses:
153153

154154
`ci_verify.py` counts the CTest rows that actually **ran** and fails below a
155155
floor — `KERNEL_MIN_TESTS` ci_verify.py:320 and `RELEASE_MIN_TESTS`
156-
ci_verify.py:587.
156+
ci_verify.py:594.
157157
A deleted or silently skipped row is a failure, not a quieter run. If your
158158
change adds rows, raise the floor in the same commit and say by how much; if it
159159
legitimately removes one, lower it deliberately and say why. `--min-tests`

‎docs/adr/0001-kernel-adapter-boundary.md‎

Lines changed: 17 additions & 16 deletions
Large diffs are not rendered by default.

‎docs/design/native-feature-parity.md‎

Lines changed: 35 additions & 35 deletions
Large diffs are not rendered by default.

‎docs/native-refactor-progress.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,7 @@ repository and were never pushed.
183183
| Compact journal records and a history ring buffer (L3) | Smaller command records and a ring buffer for the request core's history. `WorkingRequestCore::history` (`include/pineforge/native_order.hpp:1709`) hands out the whole vector and the core holds `history_` (`include/pineforge/native_order.hpp:2197`) inside `native_order_v7`, so both are a layout and API change | L3, spark A725, `perf` call graphs, inclusive shares: at most 3.68 % of bracket, 11.99 % of re-issue, 4.26 % of market and 3.42 % of the 14 probes (retiring the history alone: 1.64, 4.51, 2.53 and 1.00 %); only part is realisable, since records must still be written | "R5 L3 DONE + ACCEPTED 2026-09-24 22:10 Taipei", ruling 1: "DEFERRED to the PERF-D3 structural plan"; TRIAGE1 §D: "compact journal records" | `r5/l3` (`b33ea659`, merged by INT22); `$EV/exec/L3-scratch/`; spark `~/l3/prof` | A `native_order` layout epoch |
184184
| A checked-values handoff for executions (L3) | `execution_values` runs twice, in `check_execution` (`src/native_order.cpp:4791`) and again in `apply_execution` (`src/native_order.cpp:4805`); handing the checked values across needs a public type, an API addition with no layout change | L3, as above: 2.21 % of bracket for the two together, about 1.1 % recoverable | L3 ruling 2: "a public checked-values handoff for executions (~1.1% bracket) -> DEFERRED" | As above | An owner decision on the API addition |
185185
| A chunked `trades_` store (L3) | Closed trades in chunks rather than one vector; `trades_` (`include/pineforge/engine.hpp:546`) is protected `BacktestEngine` state, so script ABI | L3, as above: on bracket, vector growth 0.64 %, `Trade` moves 0.74 % and `record_close_trade` 0.43 % (at most about 1.4 %); at most 0.35 % on the 14 probes | L3 ruling 4: "chunked trades_ store (ABI) -> DEFERRED" | As above | An `engine_script_run` epoch that changes the engine's protected layout |
186-
| A zoned chart-day memo (D2-D finding 4) | On a chart with a timezone, `libc_chart_day_key` (`src/source/pine_adapter.cpp:14869`) takes the process timezone lock through `ScopedTimezone` and calls `localtime_r` on every bar, because `on_bar_open` (`src/source/pine_adapter.cpp:18134`) reads `chart_day_key` (`src/source/pine_adapter.cpp:18323`) unconditionally. The non-UTC monthly Sharpe/Sortino walk, `month_key_local` (`src/engine_metrics.cpp:33`), pays one `localtime_r` per equity point. UTC charts use D2-D's memo, `civil_chart_day_key` (`src/source/pine_adapter.cpp:14853`) | Not measured on a zoned chart. The UTC path cost 134 instructions a bar before D2-D's memo and 68 after (D2-D, spark, callgrind on the floor). PERF-K1: `ScopedTimezone` copies the zone name, which allocates for names of 16 or more characters under libstdc++ | TRIAGE1 §D: "D2-D finding 4 (the timezone `localtime_r` memo)", under the owner's cap. D2-D's report: an exact zoned memo "would need the zone's transition table, which belongs in `timezone.cpp`" | None; the UTC memo's witness is `tests/test_chart_day_memo.cpp` | A lane that owns `src/timezone.cpp` and builds an exact zone-transition memo (PERF-K1's Sitka case shows why a shortcut is not exact), byte-identical, with the zoned keys of `test_chart_day_memo` still equal to the computed ones |
186+
| A zoned chart-day memo (D2-D finding 4) | On a chart with a timezone, `libc_chart_day_key` (`src/source/pine_adapter.cpp:14869`) takes the process timezone lock through `ScopedTimezone` and calls `localtime_r` on every bar, because `on_bar_open` (`src/source/pine_adapter.cpp:18140`) reads `chart_day_key` (`src/source/pine_adapter.cpp:18329`) unconditionally. The non-UTC monthly Sharpe/Sortino walk, `month_key_local` (`src/engine_metrics.cpp:33`), pays one `localtime_r` per equity point. UTC charts use D2-D's memo, `civil_chart_day_key` (`src/source/pine_adapter.cpp:14853`) | Not measured on a zoned chart. The UTC path cost 134 instructions a bar before D2-D's memo and 68 after (D2-D, spark, callgrind on the floor). PERF-K1: `ScopedTimezone` copies the zone name, which allocates for names of 16 or more characters under libstdc++ | TRIAGE1 §D: "D2-D finding 4 (the timezone `localtime_r` memo)", under the owner's cap. D2-D's report: an exact zoned memo "would need the zone's transition table, which belongs in `timezone.cpp`" | None; the UTC memo's witness is `tests/test_chart_day_memo.cpp` | A lane that owns `src/timezone.cpp` and builds an exact zone-transition memo (PERF-K1's Sitka case shows why a shortcut is not exact), byte-identical, with the zoned keys of `test_chart_day_memo` still equal to the computed ones |
187187
| Thread-local reads that remain per call | D2-C moved the three scopes' writes onto the pump's block, `pump_ambient` (`src/native_execution_consumer.hpp:444`). A reader still reaches the block through `tl_runtime_ambient` (`src/ta_extremes_volume.cpp:29`): `bar_context` (`src/ta_extremes_volume.cpp:82`), read by every `ExtremeRing::update` (`src/ta_extremes_volume.cpp:117`), which backs `ta.highest`, `ta.lowest`, `ta.highestbars` and `ta.lowestbars` and, through an embedded `Highest` and `Lowest`, `ta.stoch`, `ta.wpr` and `ta.range`; and `matching_day_partition` (`src/timeframe.cpp:605`), read by the default-anchored `ta.vwap` through `session_day_index`, by `crosses_boundary` on a DAY period (`timeframe.change("D")`) and by the session-period helpers. `decompose_ms_local` (`src/session_time.cpp:250`) reads two thread-locals per call on a zoned clock, and the prepared order storage's `free_blocks` (`src/native_order.cpp:605`) one per prepared mutation. `ema_na_warmup_flag` is read once per EMA, at its first compute | One thread-local access per reader call. AUDIT4, macOS arm64, a dlopen'd tutorial-MACD probe, counted by `_tlv_get_addr` breakpoints and by descriptor patching: 1.000 a bar for one `ta::Highest` and for one default `ta::VWAP`, 1.999 for one `pine_hour` on a New York clock, 0.000 on the no-order floor and with orders. On Linux a TLS-descriptor call costs about 20 instructions: D2-C's 13 → 4 calls a bar was 260 → 80 instructions (spark aarch64, GCC 13, callgrind). INT23: 2.00 a bar on the zoned probe 064, 0.08 on probe 008 | None for the TA and day-partition readers: "RULING: INT23 ACCEPTED" (ledger, 2026-09-24 23:30 UTC) reads "dlopen TLS 13->0/bar", and INT23's report names only the timezone and order-core residuals. This row is their record | `$EV/exec/AUDIT4-opus-scratch/perf/tls/` | A byte-identical lane that hands these readers the pump's block without a thread-local read. `ta::bar_context()` is script ABI, so its public declaration stays as it is |
188188
| "audit-off mode", "LTO", "data-oriented layout" | Named in "OWNER DECISION 2026-09-25 00:20 Taipei" among the post-release structural items: "S1 bulk quiet bars, S2 per-strategy specialization, audit-off mode, LTO+PGO, data-oriented layout". No engine or campaign document defines, designs or measures the three | None | The owner decision | None | Struck from this inventory until a design note defines one of them |
189189

‎docs/pages/contributing-llm.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ finding to report, not a step to take.
118118

119119
10. **A test row never silently disappears.** Each profile counts the rows that
120120
*ran* against a floor: `KERNEL_MIN_TESTS` ci_verify.py:320 and
121-
`RELEASE_MIN_TESTS` ci_verify.py:587. Adding rows means raising the floor
121+
`RELEASE_MIN_TESTS` ci_verify.py:594. Adding rows means raising the floor
122122
in the same commit.
123123

124124
## The recipe for a lane
@@ -264,7 +264,7 @@ measurement that produced it, so a later change to it is visible as a change to
264264
the record, not as an edit to a literal.
265265

266266
**floor** — the minimum number of CTest rows a profile must actually run
267-
(`KERNEL_MIN_TESTS` ci_verify.py:320, `RELEASE_MIN_TESTS` ci_verify.py:587). It
267+
(`KERNEL_MIN_TESTS` ci_verify.py:320, `RELEASE_MIN_TESTS` ci_verify.py:594). It
268268
counts rows that ran, so a skipped row does not pad it.
269269

270270
**receipt** — the recorded evidence an ABI-comparison row needs (a prepared

‎scripts/ci_verify.py‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -584,7 +584,14 @@
584584
# +1 XAU-CAL test_symbol_calendar_tapes (the bar after the current one on
585585
# TradingView's symbol calendar)
586586
# No release row skips, so 790 registered is 790 run.
587-
RELEASE_MIN_TESTS = 790
587+
# 792 = those 790 plus lane FIX-E1E2's two source-bound rows, counted the
588+
# same way (the kernel profile builds neither):
589+
# +1 FIX-E1E2 test_report_timeframe_ratio (the report's script_tf_ratio
590+
# and needs_aggregation on every run route)
591+
# +1 FIX-E1E2 test_aggregated_admission_receipt_tapes (an aggregated
592+
# chart's admission receipts in their command's bar space)
593+
# No release row skips, so 792 registered is 792 run.
594+
RELEASE_MIN_TESTS = 792
588595
# ADR-0001 ruled-count floors, beside the ctest floors (R5 lane H-DOCGATES,
589596
# AUDIT4-opus X13 / docs-a N7). check_kernel_residuals.py counts the rulings
590597
# its vocabulary reads -- 174 identifiers and 45 texts on the lane's tree: the
@@ -625,9 +632,10 @@
625632
# counted the same way; 767/767/776 with lane TAIL-I's row of
626633
# RELEASE_MIN_TESTS, counted the same way;
627634
# 771/771/780 with lane XAU-CAL's four rows of RELEASE_MIN_TESTS in each,
628-
# counted the same way.
635+
# counted the same way; 773/773/782 with lane FIX-E1E2's two rows of
636+
# RELEASE_MIN_TESTS in each, counted the same way.
629637
# An excluded run must still discover at least this many rows before -LE.
630-
EXCLUDED_REGISTERED_MIN = {'debug': 771, 'sanitizers': 771, 'native': 780}
638+
EXCLUDED_REGISTERED_MIN = {'debug': 773, 'sanitizers': 773, 'native': 782}
631639
# The ctest stage's bound. A full sanitizers run (push to main, a manual
632640
# dispatch, the maintainers' verification) ran out of its 30 minutes twice on
633641
# main's four-core runner before every row had finished, so it gets an hour; a

‎scripts/test_ci_verify.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1355,7 +1355,7 @@ def test_ctest_label_exclusion_is_forwarded(self):
13551355

13561356
def test_pr_exclusion_proves_registered_minus_labelled_equals_ran(self):
13571357
self.assertEqual(ci_verify.EXCLUDED_REGISTERED_MIN,
1358-
{'debug': 771, 'sanitizers': 771, 'native': 780})
1358+
{'debug': 773, 'sanitizers': 773, 'native': 782})
13591359
for profile, registered in ci_verify.EXCLUDED_REGISTERED_MIN.items():
13601360
with self.subTest(profile=profile):
13611361
code, summary, scripted, _ = self.run_profile(

‎src/source/pine_adapter.cpp‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17050,6 +17050,8 @@ void PineExecutionAdapter::source_batch_end() {
1705017050
cap.source_batch_end();
1705117051
}
1705217052

17053+
// `bar` is the projection bar (projection_bar_index), the bar space of the
17054+
// command observations the review receipts name.
1705317055
void PineExecutionAdapter::record_market_review(
1705417056
admission::Checkpoint checkpoint, int bar,
1705517057
const std::vector<native_order::RequestHandle>& handles) {
@@ -17156,8 +17158,12 @@ void PineExecutionAdapter::refresh_pending_sizing_after_margin(
1715617158
const auto& origin = snapshot.market_admission.observation();
1715717159
auto allocation = admission_journal.reserve();
1715817160
admission::SizingEvent sizing;
17161+
// A receipt's bar is in its command's bar space: the observation
17162+
// bound the row's projection bar, the input slot, which an
17163+
// aggregated run keeps apart from the script-bar interval_index
17164+
// (lane FIX-E1E2: a receipt at interval_index preceded its command).
1715917165
sizing.receipt = {allocation.sequence(), cause_fill,
17160-
context.coordinate.interval_index, origin->command};
17166+
projection_bar_index(context), origin->command};
1716117167
sizing.incarnation = handle.incarnation;
1716217168
sizing.before = before;
1716317169
sizing.after = {snapshot.sizing.frozen_units, snapshot.sizing.equity,
@@ -17355,9 +17361,9 @@ void PineExecutionAdapter::apply_open_market_admission(
1735517361
review_handles.reserve(market.size());
1735617362
for (const auto& candidate : market) review_handles.push_back(candidate.handle);
1735717363
record_market_review(admission::Checkpoint::DefaultGross,
17358-
context.coordinate.interval_index, review_handles);
17364+
projection_bar_index(context), review_handles);
1735917365
record_market_review(admission::Checkpoint::ExplicitPair,
17360-
context.coordinate.interval_index, review_handles);
17366+
projection_bar_index(context), review_handles);
1736117367
for (const auto& handle : cancellations) {
1736217368
const auto result = require_host().cancel(handle);
1736317369
if (result.status == native_order::CancelStatus::Cancelled) retire(handle);
@@ -18064,7 +18070,7 @@ void PineExecutionAdapter::apply_terminal_explicit_market_policy(
1806418070
}
1806518071
}
1806618072
record_market_review(admission::Checkpoint::TerminalGross,
18067-
context.coordinate.interval_index,
18073+
projection_bar_index(context),
1806818074
terminal_review_handles);
1806918075

1807018076
// ab9714be pine_fills.cpp:3023-3270 and pine_orders.cpp:193-276:

‎src/source/pine_scheduler_native.cpp‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,8 @@ void PineScheduler::run_begin(PineStrategyHost& host) {
126126
const int ratio = undetected || !state.spec ? 1
127127
: tf_ratio(state.spec->input_tf, state.spec->script_tf);
128128
const bool needs_aggregation = ratio > 1 || ratio == -1;
129+
host.diag_script_tf_ratio_ = ratio;
130+
host.diag_needs_aggregation_ = needs_aggregation;
129131
input_script_completes_.assign(retained_.bars.size(), 1U);
130132
input_script_boundary_completes_.assign(retained_.bars.size(), 0U);
131133
// Which input bars complete a script bar, and which complete the previous

‎src/source/pine_strategy_host.cpp‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -528,6 +528,16 @@ void source::PineStrategyHost::on_native_applied(
528528
if (lot.entry_incarnation == event.handle().incarnation)
529529
lot.time = script_open;
530530
}
531+
// A later execution of the same pass can close this fill's lot
532+
// before this notification arrives (the reversing second call
533+
// of a process_orders_on_close pair): the rows it booked after
534+
// this event's own still carry the kernel's instant (lane
535+
// FIX-E1E2, tests/fixtures/aggregated_admission_receipt).
536+
for (std::size_t index = event.first_trade_index + event.closed_trade_count;
537+
index < trades_.size(); ++index) {
538+
if (trades_[index].entry_incarnation == event.handle().incarnation)
539+
trades_[index].entry_time = script_open;
540+
}
531541
}
532542
for (std::size_t i = 0; i < event.closed_trade_count; ++i) {
533543
const std::size_t index = event.first_trade_index + i;

0 commit comments

Comments
 (0)