Skip to content

engine: ABI v4 — live-runtime surface (flags default off, accessors, per-bar broker-state hash) - #218

Merged
luisleo526 merged 41 commits into
mainfrom
live/abi-v4
Sep 8, 2026
Merged

luisleo526 merged 41 commits into
mainfrom
live/abi-v4

Conversation

@luisleo526

Copy link
Copy Markdown
Collaborator

Implements §3 of the pineforge-live design spec (pineforge-workflow-live/docs/superpowers/specs/2026-09-07-pineforge-live-design.md).

  • flags (default off): realtime_tail, probe_suppress_tail_logic, path_order, broker_state_hash_recording, request_abort / last_run_status
  • accessors: pending-order POD mirror (pf_pending_order_v1_t, layout-described) + derived fill qty / level resolution / effective levels; closed-trade entry/exit ids, exit comment, close cause; position size / avg price / cycle seq, trail best price; current equity; script bars processed
  • pf_report_t: per-script-bar broker_state_hash array (ABI v4, append-only; 24 new exports, 56 total)
  • L0 evidence lanes: flags-off identity (312/312 corpus probes byte-identical to the branch-point engine, scripts/live_flags_off_identity.py), live-flags lane (312 probes, 0 positives, 130 open-at-end trades subtracted; seeded-positive pin), bar-identity row 1 (engine aggregate(1m) vs the graded derived 15m feed: 222,295 bars, 2 explained open divergences, 0 else)
  • docs: README symbol table (v4), docs/pages/live-surface.md, cheatsheet corrections (trail_price is read; v4 accessors; corpus slippage facts), report-schema / Rust / Python mirrors at v4

Verification: ctest 242/242; CI scripts (C-ABI runtime split, broker-state hash coverage, pending-order mirror) green; corpus sweep tiers identical to base (311 excellent, 1 anomaly).

Parity gate (whole population, experiment exp-live-abi-v4-20260908-r2, candidate snapshot 1fdc1d0c…, baseline 2a9c3d55…): hard gate 704 evaluated, 0 regressions; coverage 4190/4190; all 4190 grades identical to the baseline. The two-band target rule therefore scores 0 and the verdict is FAIL target.not-positive — a zero-delta ABI change cannot satisfy "strictly positive" as the rule stands. Merge decision and any rule amendment are the maintainer's call; no lab gate record receipt was minted.

Parked follow-ups: rename include/pineforge/pending_order_mirror.hpp → .h (pure-C header) before release; add a nonzero-slippage close_opposite_then_enter test (the cheatsheet's "masked by slippage=0" premise was false: 60/312 probes run slippage>0).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3

luisleo526 and others added 30 commits September 8, 2026 14:30
Task 0 review fix round 1: the four EXPECTED_PF_ABI mirror comments
claimed v4 doesn't change pf_report_t's layout, contradicting
pineforge.h's own v4 doxygen (broker-state hash array appended). A
later commit on this branch grows pf_report_t; left as-is, these
comments would mislead whoever extends it into thinking the ReportC
mirrors don't need to grow too, even though the version guard alone
wouldn't catch the drift. Comment-only fix, no code/behavior change.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EtGc9e9jfVe87cm7XUaQq3
…verage, minor hardening

Address task-1-review.md findings on top of 1d5d1e1:
- Test the aborted handle itself (not a fresh instance) runs to completion.
- Cover the TF-aware run() overload's own abort path on both the
  simple-loop and aggregation-loop sub-paths, reused to completion.
- Close the abort-lost window in the delegating overload by letting the
  TF-aware run() own the flag clear.
- Symmetric lookahead/aux-security cleanup in the second overload's
  AbortRequested handler.
- strategy_last_run_status(NULL) now returns -1 instead of 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
…lost window

Round 1's fix for the delegating overload's abort-lost window didn't
actually close it: the delegate it called still cleared the flag a
second time at its own entry, after the delegating overload's own
setup work. Split the TF-aware run() overload into a thin public
entry (clears last_error_/last_run_status_/abort_requested_ once)
and a private run_tf_impl() that does the actual work and never
touches those fields. All three public run() overloads now clear the
flag exactly once, at their own entry, and delegate to run_tf_impl
directly so nothing downstream clears it again.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
…rage, degenerate-TF session fallback, doc fixes

Adds TF-aware (run_simple_bar_loop) off/on test coverage for
session.islastbar and the run_tf_impl last_bar_index_/last_bar_time_
freeze, which the single-TF-only brief test could not discriminate.
Asserts last_bar_time_ directly (previously write-only) and open_profit
on the tail equity point. Aligns run_simple_bar_loop's degenerate
script_tf_seconds_<=0 fallback with engine_stream.cpp, and fixes two
stale/self-contradictory doc comments.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GjK8xqo1kMhkEvnr1ZWxHV
…ST test discrimination (fix round 1)

Task 4 review fix round 1: snapshot the dual-entry-stop arbitration
(last_bar_dual_entry_decision_) so the accessor survives a same-bar fill,
a declined stop-entry admission, or the POOC second pass — none of which
should erase the fact that an arbitration happened. Parameterise the
path-order test on the touch bar so HIGH_FIRST is actually discriminated
from AUTO, and add positive/stale-value coverage for the new accessor.
Also: PathOrderScope restores the prior override and is a single
file-scope helper, set_path_order clamps to {0,1,2}, the magnifier's
duplicated high-first expression now routes through
internal::bar_path_uses_high_first, and NULL returns -1 from
strategy_last_bar_dual_entry_path.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_019xxoremsR3SEbgw5wD7KVL
…rding, self-describing JSON

Waives broker_state_hash_recording_/broker_state_hashes_ in the §3.4
coverage gate (Critical), records the hash on the realtime stream's
script-bar dispatch site so strategy_stream_fill_report satisfies the same
len == script_bars_processed / array[last] == broker_state_hash() promise
as the three run() loops (Important), adds a stream-path test that fails
red without the fix plus an aggregated-run coverage test (Minors), and
makes run_strategy.py's --broker-state-hash JSON self-describing
(script_bars_processed + per-entry time_ms/hex hash) with a stderr warning
instead of a silent empty file when the .so predates the export.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01VFzaaNARFFoEkbhvkBQRUQ
…_* accessors (ABI v4)

scripts/gen_pending_order_mirror.py reflects struct PendingOrder out of
engine.hpp (comment-stripped, wrapped initialisers joined, every
declaration must be exactly `TYPE NAME [= init];` or generation fails)
and emits the C-only include/pineforge/pending_order_mirror.hpp
(pf_pending_order_v1_t: struct_version=1, size, then all 98 members --
scalars/enums by value, std::string as char[64] + _truncated + FNV-1a
_hash64 of the full string; pf_field_desc_t) and
src/pending_order_mirror.cpp (fill_pending_order_mirror,
pending_order_layout). --check is a CI step and a ctest, so PendingOrder
cannot grow a member nobody mirrored or waived
(scripts/pending_order_mirror_waivers.txt, currently empty).

pineforge.h gains strategy_pending_orders_len, strategy_pending_order_get
(min(size_in, sizeof) prefix copy; -1 on NULL/out-of-range) and
strategy_pending_order_layout (static self-describing field table);
BacktestEngine gains pending_order_count()/pending_order_at().
run_strategy.py builds its PendingOrderV1 ctypes struct from that table
(refusing any offset/size disagreement) and adds --dump-book PATH.

Tests: tests/test_live_pending_order_mirror.cpp (fill semantics, layout
table, prefix/over-size copy contracts, accessors),
scripts/test_run_strategy_dump_book.py (layout-built reader). Pure add;
historical runs byte-identical; ABI append-only.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01DLL2ZrWggkJhn8aws5ccao
…PyramidEntry::entry_incarnation (task-5 carried ruling)

scripts/check_broker_state_hash_coverage.py now also loads the reflected
PendingOrder member list (gen_pending_order_mirror.members) and requires
every member to be hashed as o.<name> inside broker_state_hash()'s
pending_orders_ loop or waived as pending_order.<name> in
scripts/broker_state_hash_waivers.txt.

broker_state_hash() gains the four fill-deciding comparands named by the
task-5 re-review (rounded_signal_cost_close_only,
created_after_position_close_in_bar, signal_close_mc_remaining_qty,
created_position_side) plus the 31 other PendingOrder members whose own
doc names the fill/admission/eligibility path that reads them -- no
honest "cannot change a future fill" waiver existed for them -- and
PyramidEntry::entry_incarnation in the pyramid loop. `comment` is the
one waiver (trade-report label, never read by fill logic). Only the hash
VALUE changes; no engine behaviour path is touched.

tests/test_live_state_hash.cpp: 36 new mutation pins, one per newly
hashed field.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01DLL2ZrWggkJhn8aws5ccao
…, hex every uint64, generator/coverage anchoring, size_in < 8 rejection

Important 1: scripts/test_run_strategy_dump_book.py drives
Strategy.read_pending_orders' real loop against a stub lib (a fake
strategy_pending_orders_len / strategy_pending_order_get that memmoves
prepared PendingOrderV1 records): happy path over two records in book
order, empty book, and the three refusals (struct_version != 1, size !=
layout-built sizeof, get() != 0). 16 tests; already a ctest.

Minor 2: pending_order_to_dict hexes EVERY uint64_t field (incarnation,
*_incarnation, signal_close_mc_fill_seq, the _hash64s), keyed on the
layout type, and says so.
Minor 4: gen_pending_order_mirror.struct_body strips comments before
anchoring on `struct PendingOrder {` and fails unless it occurs exactly
once; generated output unchanged (--check up to date).
Minor 5: check_broker_state_hash_coverage requires exactly one
`for (const auto& o : pending_orders_) {` loop, else exits 2.
Minor 6: strategy_pending_order_get returns -1 with nothing written when
size_in < 8 (cannot hold struct_version + size); documented in
pineforge.h; C++ test covers size_in 0/7 (-1, buffer untouched) and 8.

Verification: gen --check, hash coverage, c_abi runtime check, targeted
ctest 5/5, full ctest 235/235.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01DLL2ZrWggkJhn8aws5ccao
…s + position scalars for the live runtime

ABI v4 live-runtime surface (task 8, spec 3.6). Pure const reads of the
engine's own sizing / admission / level-resolution predicates so the live
runtime never re-implements them:

- BacktestEngine::probe_fill_qty(index, fill_price, &qty, &close_only,
  &partition): the quantity the entry kernel would open at that fill, tagged
  0 EXPLICIT / 1 FROZEN_PLACEMENT / 2 DEFAULT_STOP_PLACEMENT / 3 AT_FILL --
  the zero-lot decline gate's own chain (frozen_default_qty ->
  use_default_stop_placement_qty -> calc_qty_for_type at the slipped fill),
  the MARKET kernel's frozen transactions (paired flat pair / sbmt) and the
  RAW_ORDER kernel's verbatim explicit qty; close_only = the dispatch sites'
  affordability_close_only | prior_cycle_close_only |
  same_cycle_frozen_tx_exact_flat | paired flat market vs an opposite
  position, each spelled as apply_entry_order_fill / apply_market_order_fill
  spell it. rc 1 for an EXIT (no opening size).
- pending_order_level_resolved: the materialize_relative_exit_prices_for_
  live_position / eligibility gate (from_entry filled this cycle).
- pending_order_effective_levels: set legs verbatim; tick offsets resolved
  against position_entry_price_ with the POSITION side's sign exactly as the
  fill path (limit = entry + dir*profit_ticks*mintick, stop = entry -
  dir*loss_ticks*mintick, level_on_price_grid; trail activation via
  trail_points_to_ticks + snap_trail_level_to_tick_grid, trail_points over
  trail_price); NaN where unset / unresolvable.
- position_avg_price / position_cycle_seq / trail_best_price.

C ABI: strategy_pending_order_fill_qty / _level_resolved /
_effective_levels, strategy_trail_best_price, strategy_position_avg_price
(NaN when flat), strategy_position_cycle_seq -- -1 / NaN on a NULL handle,
full pf_live doxygen (partition codes, signs, NaN semantics);
EXPECTED_RUNTIME += 6. run_strategy.py: hasattr-guarded prototypes,
read_order_derived / read_position_scalars, --dump-book gains per-order
`derived` (fill qty probed at each finite effective stop/limit level and at
the last close; level_resolved; effective_levels) and `position`.

Tests: tests/test_live_order_derived.cpp pins all four partitions
(explicit 2.0; frozen percent 100; default-percent stop 99 = floor(1e4/101),
falling back to AT_FILL on a zero print; FIXED-at-fill 3; RAW_ORDER 2.5
verbatim), the prior-cycle close-only rule, the 98/103 long bracket and the
97/102/99.5 short bracket + trail activation, and the C-ABI NULL contract;
scripts/test_run_strategy_dump_book.py +6 stub-lib tests for the derived
readers. Pure add: no historical-path code touched, ABI append-only.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
…e-bar-market reversal kernels in probe_fill_qty

Review task-8-review.md Important 1 (controller ruling: model, not
document). probe_fill_qty now takes apply_market_order_fill's two reversal
kernels ahead of the generic chain, in the kernel's own order:

- short_seed_collision_final_short_is_live: the final short closes both
  physical LONG lots and re-opens SHORT the residual pyramid_entries_[0].qty
  - pyramid_entries_[1].qty iff > kQtyEpsilon (finding 272) -> qty =
  residual, partition 1, close_only = residual <= eps.
- apply_same_bar_market_tx_reversal (round 8 family S member against an
  opposite live position): close min(tx, live), open remainder tx -
  min(tx, live) iff > kQtyEpsilon -> qty = remainder, partition 1,
  close_only = remainder <= eps. The kept-over-cap add and the flat frozen
  transaction keep their sbmt_tx_qty dispatch.

Doxygen (engine.hpp + pineforge.h): both kernels described next to the
tv_carry_qty paragraph; EXPLICIT is lot-floored for strategy.entry and
verbatim for strategy.order; close_only reads "the kernel's close-only
predicate fires" with the close_opposite_then_enter remainder caveat, the
replaced_percent_short dispatch, and the current-book probe semantics.

Tests (tests/test_live_order_derived.cpp): test_sbmt_kernels reaches all
four family-S shapes through the public API (reversal remainder 1;
reversal after a same-bar close -> 0 / close_only 1; kept-over-cap 2;
flat pair 1 AT_FILL / 2 FROZEN); test_short_seed_final_short installs the
exact two-lot book through the test subclass's protected access (the
predicate is live only inside the next bar's fill loop) and pins residual
2 (L 3, S 1), 0 / close_only 1 (L 1, S 1) and the sibling's ordinary
chain; test_limit_route_slippage pins apply_slippage vs apply_limit_fill
sizing bases (CASH 1000, slippage 2: 1000/100.02 vs 10.0). <limits>
included; test B points at the meaningful AT_FILL pins.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
…s reference provenance, two-way coverage, tagged failures

Fix round 2 addresses the reviewer's nine quality findings on
task-10-review.md (spec compliance was already approved):

- F1: the pin now asserts fresh.report_trade_count() > 0 and
  reused.report_trade_count() > 0 (after the 50-bar run, before the
  400-bar rerun), so a future engine change that stops filling would fail
  loudly instead of leaving same_trades() vacuously true on two empty
  lists.
- F2/F6: --reference mode now diffs the reference dir's slugs against the
  slugs actually produced this run; anything present only in the
  reference (e.g. a probe whose library failed to build on this branch)
  is tagged MISSING instead of being silently absorbed into "0 differ".
  Every failure is tagged DIFF/FAIL/MISSING with separate counts printed;
  "M differ" in the summary line counts DIFF only. The count of
  source-marked probe dirs with no compiled library is now always
  printed, never silently skipped.
- F3: --emit DIR now writes DIR/provenance.json (root, engine_head,
  corpus_head, per-slug compiled-lib sha256), refuses to write into a
  non-empty DIR unless --force (which clears it first), and --reference
  DIR reads that file back, printing "reference engine <sha> vs current
  engine <sha>" and refusing (exit non-zero) a same-engine comparison
  unless --allow-same-engine is passed deliberately.
- F4: emit-mode failures now print the harness's real error (the last
  non-empty output line) plus a first-3-full-details block, instead of
  the constant "run_strategy.py failed:" label that hid the actual
  exception.
- F5: --only now matches the probe path relative to --root, not the
  absolute checkout path (which could match unrelated substrings of the
  checkout's own directory name).
- F7: the docstring keeps only the timeless rule; the dated episode below
  moves out of the script.
- F8: default mode (neither --emit nor --reference) prints a one-line
  banner that the committed corpus reference can lag the engine, pointing
  at the three-command procedure.
- F9: reworded the "small crossover" comment -- the strategy is a
  modulo-bar_index entry/exit schedule, not a crossover.

Dated episode moved out of the script docstring per F7: fix round 1's
committed corpus/validation/<slug>/engine_trades.csv reference was found
stale -- generated by a7cb5b6, 193 commits behind this branch's own fork
point -- so the authoritative L0 comparison was redefined as this
branch's flags-off trades against the branch-point engine e3f0d28 (the
merge-base with origin/main), not the committed CSVs.

Verified: ./build/bin/test_live_flags_off_identity -> OK (0 failures);
ctest -R live_flags -> 1/1 passed; reference re-emitted against e3f0d28
with --force (provenance.json: engine_head e3f0d28..., corpus_head
b46cd80c..., 312 libs); --reference build/ref_trades_e3f0d28 ->
"live_flags_off_identity: 312 probes, 0 differ (reference engine
e3f0d28)" / "0 DIFF, 0 FAIL, 0 MISSING", exit 0; the same-engine
guard demonstrated non-zero exit comparing --root . against its own
--emit; the MISSING tag demonstrated non-zero exit on a synthetic
partial reference; git -C corpus status --porcelain clean throughout.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018trdMSvtzTBJgwNcjmksvP
…rovenance corpus_head is checked

N1: --reference --only now filters ref_slugs with the same substring
rule as find_cases (match against corpus/validation/<slug>), so a
spot-check no longer drowns in spurious MISSING rows for every probe
--only excluded from the run side.

N2: --reference now reads provenance.corpus_head, prints it beside the
engine-identity line, and exits 1 if it differs from the current
root's corpus submodule HEAD (no override flag) — a reference emitted
against a different corpus commit is never a meaningful L0 comparison.

Verified: --reference build/ref_trades_e3f0d28 --only ta-sma-152 --jobs 2
-> 1 probes, 0 differ, 0 MISSING, exit 0. Full run --reference
build/ref_trades_e3f0d28 --jobs 8 -> 312 probes, 0 differ, exit 0,
corpus clean. A corpus_head mismatch (hand-edited provenance.json)
correctly refuses with exit 1.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
… labelled for the admission lint

scripts/live_flags_lane.py reruns every compiled corpus probe through
run_strategy.py three times (flags-off, --realtime-tail N+1000,
--realtime-tail 2N where N is the probe's feed row count) and labels a
probe positive iff a trade row strictly before the feed's own final bar
(interior) differs between the flags-off and a flagged run. Reuses
find_cases/run_probe (now extra_args-capable) from
live_flags_off_identity.py and inputs_run_kwargs/DEFAULT_OHLCV/
_fmt_time_utc from run_strategy.py -- no duplicated runner or CSV
parsing. Output: build/live_flags_lane.json.

Result: 312 probes, 130 positives, 0 errors, 41s wall (--jobs 8).

IMPORTANT caveat carried in the report (task-11-report.md): all 130
positives were individually verified to reduce to ONE mechanical cause
-- run_strategy.py's range-end forced close (a position still open at
TradingView's declared range end, marked "open" in the Engine
range-end CSV column) is skipped when --realtime-tail is on (as
documented for strategy_set_realtime_tail), so that synthetic trade's
Entry+Exit rows vanish entirely; the Entry leg is frequently dated well
before the final bar, so it counts as "interior" under the ruled
per-row definition even though no script logic changed. Verified via a
fresh base rerun of all 130: 130/130 show the vanished trade's Exit
marked "open". Only 2/130 (mtf-htf-60-close-change-baseline-01,
mtf-htf-60-close-roll-01) contain any of last_bar_index/
barstate.islast/barstate.isrealtime/barstate.isconfirmed/timenow in
source, and even those two show the identical mechanical artifact, not
a distinguishable effect of their barstate.islast branch. 0/130 are
currently evidenced as genuine live-state-dependent trading; the
positive set is not yet usable as admission-lint ground truth as
defined. differs_any_by_horizon (context, ruling 6) == positives
exactly (130 == 130) rather than a superset, because the ruling's
expectation that this artifact would only ever be non-interior turned
out not to hold.

Positive slugs (P=130):
analyzer-anvil-percent-costs-01
analyzer-cashflow-cash-costs-01
bracket-atr-trailing-stop-state-01
composite-4emarsi-quad-ema-stack-01
composite-4emarsi-rsi-pullback-latch-01
composite-boscurv-integration-01
composite-boscurv-linreg-slope-channel-01
composite-boscurv-pivot-bos-trigger-01
composite-ies-bb-kc-squeeze-release-01
composite-ies-pivot-liquidity-sweep-01
composite-ies-pressure-gauge-01
composite-ies-rsi-macd-momentum-01
composite-ies-three-ema-bias-score-01
composite-kanuck-calc-on-every-tick-01
composite-kanuck-integration-01
composite-kanuck-kama-state-recurrence-01
composite-kanuck-max-bars-back-500-01
composite-kkb-ema-atr-breakout-band-01
composite-kkb-integration-01
composite-kkb-kalman-filter-1d-01
composite-kkb-margin-100-pct-01
composite-liqsweep-integration-01
composite-liqsweep-pivot-hh-ll-01
composite-liqsweep-wait-one-continuation-01
composite-liqsweep-wick-pierce-close-back-01
composite-marketshift-integration-01
composite-marketshift-pivot-state-machine-01
composite-marketshift-rolling-highest-lowest-01
composite-marketshift-state-edge-detector-01
composite-scalping-fast-ma-cross-trigger-01
composite-trendmaster-line-new-projection-01
composite-trendmaster-pivot-anchored-bracket-01
composite-trendmaster-three-tier-ema-state-01
composite-trendmaster-trend-momentum-structure-gate-01
composite-vcp-rsi-smooth-divergence-01
composite-wunderscalper-alert-templates-01
composite-wunderscalper-explicit-reverse-01
composite-wunderscalper-integration-01
drawing-chartpoint-trendline
enum-relay-regime-01
input-source-runtime-override-high-01
input-source-subscript-hl2-01
ltf-bool-array-bull-majority-01
ltf-numeric-float-ratio15-01
matrix-covariance-eigen-pca-01
mtf-cascade-tuple-01
mtf-daily-array-median-percentrank-01
mtf-daily-ema26-warmup-01
mtf-htf-60-close-change-baseline-01
mtf-htf-60-close-roll-01
mtf-htf-60-gaps-on-roll-01
mtf-htf-60-rsi14-inside-security-01
mtf-htf-60-sma20-inside-security-01
mtf-htf-weekly-sma-cross-01
mtf-orbit-trend-01
mtf-rangefinder-daily-reference-01
mtf-roll-state-60-240-d-minimal-01
mtf-triple-tf-close-confluence-01
mtf-triple-tf-macd-hist-confluence-01
na-nz-fixnan-history-chain-01
order-cross-entry-cancel-same-pass-01
order-deferred-flip-guaranteed-gap-stops-01
order-dual-stop-cancel-rotation-01
order-harbor-stop-limit-01
order-keystone-limit-replace-01
order-range-expansion-pending-stop-01
order-same-id-stop-raise-only-01
order-stale-stop-after-close-no-cancel-01
order-stop-cancel-no-regime-close-01
order-stop-entry-cancel-opposite-01
order-switchback-all-in-reversal-01
order-tranche-partial-market-close-01
stats-eventrades-zero-pnl-count-01
stats-ledger-outcome-throttle-01
ta-accdist-ema-cross-01
ta-bb-rsi-mean-reversion-01
ta-cci-threshold-cross-01
ta-chandelier-exit-direction-01
ta-closedtrades-risk-introspection-01
ta-cmo-9-zero-cross-01
ta-cog-10-signal-cross-01
ta-dmi-adx-di-cross-01
ta-donchian-channel-breakout-01
ta-dual-ma-switch-dispatch-01
ta-dual-thrust-open-anchored-range-01
ta-ema-ribbon-stack-transition-01
ta-engulfing-candle-pattern-01
ta-highestbars-lowestbars-breakout-01
ta-hma-55-close-cross-01
ta-hma-fast-slow-cross-01
ta-inside-bar-engulfing-01
ta-kama-style-efficiency-ratio-01
ta-macd-12-26-9-line-signal-cross-01
ta-macd-histogram-reversal-01
ta-map-regime-threshold-lookup-01
ta-median-vs-ema-cross-01
ta-mfi-14-bands-20-80-01
ta-momentum-roc-zero-cross-01
ta-multi-indicator-score-composite-01
ta-nvi-pvi-cross-01
ta-obv-ema-cross-01
ta-pivot-array-unshift-pop-01
ta-pivot-confirmed-break-01
ta-pivot-point-levels-break-01
ta-pvt-ema-cross-01
ta-range-filter-var-band-01
ta-rci-14-zero-cross-01
ta-rsi-ema-signal-cross-01
ta-rsi14-bands-30-70-01
ta-rsi14-cross-50-01
ta-sar-flip-entry-01
ta-sma-152-close-cross-01
ta-sma-dual-cross-01
ta-stdev-sma-expansion-break-01
ta-stoch-slow-k-d-cross-01
ta-stochastic-rsi-cross-01
ta-supertrend-adx-filter-01
ta-supertrend-direction-flip-01
ta-triple-sma-stack-latch-01
ta-tsi-25-13-signal-cross-01
ta-vwma-vs-sma-divergence-01
ta-wpr-14-bands-01
timeframe-dial-main-period-01
timeframe-main-period-self-adaptive-01
udt-method-drives-strategy-entry-01
udt-method-in-if-else-branch-01
udt-method-in-switch-arms-01
udt-method-tuple-return-destructure-01
udt-regime-stack-stress-01
udt-switchboard-method-arms-01

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
…t carries provenance.json

N3: --emit DIR --force previously rmtree'd ANY non-empty DIR -- a typo
like --emit build --force would wipe the whole build tree. Now --force
only clears DIR when DIR/provenance.json exists (i.e. it looks like a
prior --emit output); otherwise it refuses, naming the missing
provenance file, even with --force.

Verified: a non-empty junk dir with no provenance.json refuses under
--force (exit 1, dir untouched); a real prior --emit dir still clears
and re-emits under --force (exit 0). Full corpus clean afterwards.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
… labelled for the admission lint

scripts/live_flags_lane.py reruns every compiled corpus probe through
run_strategy.py three times (flags-off, --realtime-tail N+1000,
--realtime-tail 2N where N is the probe's feed row count), all three
with --broker-state-hash (behaviour-neutral per Task 10's pin).

A probe is a positive iff, at either horizon:
  A. Trade-level comparison: after removing the harness's own synthetic
     range-end-close trade (Engine range-end == "open") from the BASE
     side only, the remaining rows dated before the feed's final bar
     differ, base vs flagged; or
  B. Broker-state hash-prefix check: the per-script-bar broker-state
     hash arrays (--broker-state-hash) differ anywhere before each
     side's own final (tail) bar.

Reuses find_cases/run_probe (extra_args-capable) from
live_flags_off_identity.py and inputs_run_kwargs/DEFAULT_OHLCV/
_fmt_time_utc/ENGINE_RANGE_END_COLUMN/ENGINE_RANGE_END_OPEN from
run_strategy.py -- no duplicated runner or CSV parsing. Output:
build/live_flags_lane.json.

Result: 312 probes, P=0 positives, 0 errors (~4m19s wall, --jobs 8).
open_at_end_probes=130 (the harness's own range-end-close convention,
context only, matches the prior naive rule's positive count exactly).

This replaces an earlier per-row "interior" rule (first committed as
abde983) that gave P=130: every one of those 130 was individually
verified (fresh base rerun, 130/130) to be the SAME harness artifact --
run_strategy.py's range-end forced close is skipped when
--realtime-tail is on, so a still-open position's Entry+Exit rows
vanish entirely, and the Entry leg is often dated well before the final
bar, satisfying the old per-row rule despite no script logic changing.
Only 2/130 contained any of last_bar_index/barstate.islast/
barstate.isrealtime/barstate.isconfirmed/timenow in source
(mtf-htf-60-close-change-baseline-01, mtf-htf-60-close-roll-01), and
even those two hand-check clean under the new rule: their
"if barstate.islast and strategy.position_size > 0: strategy.close(...)"
branch is a same-bar close under process_orders_on_close=false, so it
places an order that only fills next-bar-open -- which never comes in a
finite run -- making the branch an observable no-op; their
base-vs-flagged broker-state hash arrays are bit-identical at every
bar including the last, confirmed directly by check B.

Full report (mechanism analysis + hand-check) in
.superpowers/sdd/2026-09-07-plan-a-engine-abi/task-11-report.md.

Positive slugs (P=0): none.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
…lane (last_bar_index-dependent entry vanishes under realtime_tail; hash prefix diverges interior)

The corpus gives scripts/live_flags_lane.py P=0 (130 open-at-end trades
correctly subtracted, 0 hash-prefix mismatches) -- correct given the
corpus has no genuinely last_bar_index-dependent probe, but that alone
does not prove the lane's two detection checks (trade-level comparison
A, broker-state hash-prefix check B) actually fire on a real one.

tests/test_live_flags_lane_positive.cpp is a seeded engine-level pin
(registered in tests/CMakeLists.txt like test_live_flags_off_identity,
modelled on it and on test_live_realtime_tail.cpp):

- LastBarDependentStrategy enters long when pine_bar_index() ==
  pine_last_bar_index() - 5 and closes 2 bars later -- genuinely
  last_bar_index-dependent trading.
- Flags-off (N=400 bars): the trigger bar (N-6) is inside [0, N-1] ->
  >=1 trade.
- set_realtime_tail(true, 2N): pine_last_bar_index() is frozen at 2N-1
  for the WHOLE run, so the trigger bar (2N-6) never arrives within
  [0, N-1] -> 0 trades. This is exactly the mechanism
  scripts/live_flags_lane.py's check A would see as a vanished trade.
- With set_broker_state_hash_recording(true) on both runs: the two
  per-bar hash arrays are equal for every bar before the entry bar and
  differ from the entry bar onward -- interior (entry bar < N-1), not
  just the final bar. Exactly what check B is built to catch.
- Indifferent (a modulo-bar_index schedule from
  tests/test_live_flags_off_identity.cpp's Sma, never reads
  last_bar_index): identical trades and identical hashes over
  [0, N-2] between flags-off and the 2N-horizon run either way (the
  final bar's hash may differ -- the range-end close is skipped on the
  tail -- so the assertion is prefix-only, mirroring
  live_flags_lane.hash_first_diff()'s own exclusion).

Also added one paragraph to scripts/live_flags_lane.py's module
docstring stating the corpus is currently P=0 and pointing at this test
as the seeded positive that proves the mechanism.

Verified:
  $ cmake --build build -j8 --target test_live_flags_lane_positive
  [100%] Built target test_live_flags_lane_positive
  $ ./build/bin/test_live_flags_lane_positive && echo OK
  OK
  $ cd build && ctest -R live_ --output-on-failure
  100% tests passed, 0 tests failed out of 13

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
luisleo526 and others added 11 commits September 8, 2026 14:30
…), strip every range-end trade, --out + atomic write, last-processed-bar boundary, self-test

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
…ecks before the run

Task-11 re-review round 3 leftovers (task-11-rereview.md F8, New findings 1):

- scripts/live_flags_lane.py: _feed_cache was written from ThreadPoolExecutor
  worker threads with no lock. Pre-warm it sequentially in main(), calling
  feed_n_bars(probe_feed(strat_dir)) for every case before the pool starts,
  instead of adding a threading.Lock around an idempotent per-path value.

- scripts/run_strategy.py: the hard hasattr checks for
  strategy_set_realtime_tail / strategy_set_broker_state_hash_recording ran
  AFTER strat.run(...), so a stale .so burned a full backtest (minutes on the
  default feed) before exiting 1. Moved both checks to right after
  `strat = Strategy(so_path)` and before strat.run(); the receipt print and
  broker_state_hash.json write stay where they were, after the run.

Verified: `python3 scripts/live_flags_lane.py --self-test` (6/6, exit 0);
`python3 scripts/live_flags_lane.py --only ta-sma-152 --jobs 2` (exit 0);
`python3 scripts/run_strategy.py corpus/validation/ta-sma-152-close-cross-01
--so-name strategy.dylib -o build/t12_check/engine_trades.csv
--realtime-tail 444590 | grep realtime-tail` -> "realtime-tail: on
horizon=444590", exit 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
…ed derived 15m feed

tools/aggregate_feed.cpp runs the engine's own TimeframeAggregator (same
constructor as the aggregator built at src/engine_run.cpp:1472-1474 --
TimeframeAggregator(tf, "1", tz="UTC", session="24x7"), the engine's own
SymInfo defaults per include/pineforge/engine.hpp:1043-1044) over the
committed 1m OHLCV feed and writes the completed script bars.
scripts/bar_identity_lane.py diffs that output against the graded derived
15m chart feed (corpus/data/derived/ohlcv_ETH-USDT-USDT_15m.csv), field by
field, and classifies every divergence against the bucket's own 1m source
rows instead of just counting it.

Deviation from this task's brief, verified against source (flagging for
review): the brief's ruling read the engine's per-input-bar loop
(src/engine_run.cpp:1950-1971) as gating script-bar completion on
`ab.is_complete && tf_change(ab.bar.timestamp, input_bars[i].timestamp,
script_tf_, tz, session)`. Reading those lines directly: the actual
dispatch gate at :1970 (`if (ab.is_complete) { ... dispatch_bar() ... }`)
is unconditional on that `tf_change` result -- `completed_on_boundary`
(`ab.is_complete && tf_change(...)`) instead gates ONLY the
request.security publish-at-boundary path inside the
`security_eval_states_` loop at :1954-1969, a separate subsystem this
standalone tool has no equivalent of. Confirmed against the
TimeframeAggregator RATIO-mode source (src/timeframe.cpp:937-1030): for
gap-free 1m->15m aggregation, the normal count-reached completion fires on
the bucket's own last sub-bar, which is still inside that same bucket as
ab.bar, so `tf_change(ab.bar.timestamp, input_bars[i].timestamp, ...)` is
FALSE on essentially every regular completion -- gating this tool's
emission on `ab.is_complete && tf_change(...)` would silently emit almost
nothing for the ~222k regular buckets, only firing on boundary-triggered
completions (gaps/session edges), contradicting both the engine's actual
dispatch behaviour and the brief's own "~222k bars compared" expectation.
Implemented `if (ab.is_complete)` alone (matching this task's own step-1
reference implementation and its expected counts); see the extensive
comment in tools/aggregate_feed.cpp for the full trace. No trailing-partial
flush either: the engine's loop has no post-loop flush of a pending
partial bucket (has_pending_partial()/current()/last_completed() are never
called from engine_run.cpp), and empirically this corpus's 1m feed has no
genuine trailing partial anyway (its last 15m bucket already holds all 15
sub-bars) -- `trailing_partial` is still computed and published for
diagnostic value.

Counts (build/bar_identity_row1.json):
  bars_compared: 222295 (1m rows: 3334340; aggregate rows: 222295; derived rows: 222295)
  trailing_partial: 0
  divergence: {missing_in_aggregate: 0, missing_in_derived: 0, open: 2, high: 0, low: 0, close: 0, volume: 0}
  classification: {open_explained_leading_zero_volume: 2, open_unexplained: 0,
                   hl_explained_zero_volume_row: 0, high_unexplained: 0,
                   low_unexplained: 0, close_unexplained: 0}

Both open divergences (ts 1730133000000, 1756449000000) are explained: the
bucket's leading row(s) are zero-volume, the aggregate's open is the
bucket's true first row, and the derived feed's open is the first
positive-volume row's, exactly per scripts/derive_corpus_feeds.py's
documented resample rule. high/low/close/volume: zero divergences (both
aggregations fold every row's H/L/C the same way regardless of volume).
Zero unexplained divergences -- exit 0.

Verified: `cmake -B build -S .` (re-run after the CMakeLists.txt edit) then
`cmake --build build -j8 --target aggregate_feed` and the 13 `test_live_*`
+ `live_flags_lane_selftest` targets; `cd build && ctest -R live_
--output-on-failure` -> 100% tests passed, 14/14.
`git -C corpus status --porcelain` empty.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
… classifier, volume/missing count as unexplained, aggregator provenance, CLI; lane pre-warm isolation

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
…both sides, --out-csv per tf, first_skipped_line receipt

Fixes task-12-rereview's five new findings:
- N1: bar_identity_lane.py hard-fails (clean sys.exit) when aggregate_feed's
  aggregator provenance stderr line is absent or missing a key, instead of
  silently defaulting to the constants it was meant to verify against.
- N2: aggregate_feed.cpp hoists kInputTf/kTz/kSession constexpr so the
  TimeframeAggregator ctor and the provenance print can never drift apart.
- N3: both bar_identity_lane.py CSV loaders (load, and the 1m row-count/
  divergent-bucket pass) sys.exit with a clear message when the header
  lacks the six required columns, instead of a raw KeyError traceback.
- N4: --out-csv now defaults to build/aggregate_{tf}m.csv (was a fixed
  aggregate_15m.csv regardless of --tf), so concurrent --tf runs don't
  clobber each other's scratch file.
- N5: aggregate_feed records first_skipped_line (1-based) and prints it on
  the receipt line; the lane relays it in its error message so a dropped
  row is named, not just counted.

Verified: rebuilt aggregate_feed; scripts/bar_identity_lane.py reproduces
the same published counts (bars_compared 222295, open 2 explained, all
else 0, exit 0); a garbage line inserted into a scratch 1m copy under
build/ produces "first_skipped_line=500001" in the lane's error; a
header-less scratch derived copy produces a clean "has no ... header"
sys.exit instead of a traceback. live_flags_lane --self-test 6/6, ctest
live_ 14/14. Scratch files under build/ deleted.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_013ZEL52UCofp3j7gfABW7uB
…d; v4 accessors; corpus slippage facts)

README.md: symbol table and version line updated from ABI v3/32 symbols to
ABI v4/56 symbols (32 + 24, verified by counting `^PF_API ` declarations);
24 new rows for the live-runtime surface; pf_report_t's broker_state_hash
array and pf_pending_order_v1_t noted; v0.14.0 release line added.

docs/cheatsheet-runtime-and-execution.md: three corrections against the
actual code (per AGENTS.md SOP, spec item 3.9) --
- trail_price IS read by the fill path (engine_fills.cpp:4916-4926,
  8006-8019: used as the trail-activation level whenever trail_points is
  unset), not "accepted but ignored".
- the equity curve exists since ABI v2; ABI v4 adds the closed-trade
  id/exit-id/exit-comment/close-cause accessors and
  strategy_script_bars_processed; entry_bar_index/exit_bar_index were
  already struct fields on pf_trade_t pre-v4 (no "ABI vN" doc-comment
  tag), so the old "no bar_index over the C ABI" claim was already wrong
  independent of this task -- what remains genuinely unavailable is
  stated explicitly.
- corpus slippage/commission facts corrected: 252/312 probes run
  slippage=0, but 60/312 declare slippage=1|2 and 62/312 a non-zero
  commission_value -- the adjacent "masked because all probes run
  slippage=0" argument for the possible close_opposite_then_enter
  double-slip bug does not hold for those 60; flagged inline rather than
  silently kept, since whether any of them exercises that path is
  unverified (out of scope here: src/ is not touched by this task).

docs/pages/streaming.md: new paragraph stating pineforge-live does not use
the strategy_stream_* lifecycle (different fill model) and instead
recomputes run_backtest_full over script-TF bars with the ABI v4 flags;
links to the new page.

docs/pages/live-surface.md (new): the 24 v4 symbols in one table (purpose/
default/"changes history?"=No), flag semantics for realtime_tail,
probe_suppress_tail_logic, path_order and broker_state_hash_recording
matching spec sec 3.1-3.4 exactly, abort semantics (3.5), string lifetimes
and exit-id conventions, the report array, the generated pending-order
mirror, and an evidence section naming the three L0 lanes with their
published numbers (identity 312/0 differ; live-flags 312/0 positives/130
open-at-end subtracted; bar-identity row 1: 222295 bars/2 explained/0
else). Added to docs/Doxyfile's INPUT list (this repo's Doxygen nav) and
linked from docs/pages/index.md's realtime-feed tab.

Verified: check_c_abi_runtime.py, check_broker_state_hash_coverage.py,
gen_pending_order_mirror.py --check all pass; grep -c
strategy_set_realtime_tail README.md docs/pages/live-surface.md ->
README.md:1, live-surface.md:5; ctest 240/240; src/include/corpus/tests
untouched by this commit.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_013ZEL52UCofp3j7gfABW7uB
…uppress/range-end independence, trail_price fill-path citation, return conventions, unreleased entry

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01PXmkFdpbvYDBUY3txrJ2MR
…pshot reset, close_cause -1 convention, hard flag guards, hash-coverage ctest, zero-probe lane guards, ABI-4 docs

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q5XG4crQFDTpuWXSyrZWHJ
…oop (aggregation keeps first-bar extrapolation); gapped/aggregated tail tests; stream dual-entry reset pin; identity -dirty label

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Wrma782hjMzZFuuzv6YCf3
return waivers


def _pending_order_loop_body(src: str) -> str:

sys.path.insert(0, str(Path(__file__).resolve().parent))

import run_strategy # noqa: E402
@luisleo526
luisleo526 merged commit 41c9c74 into main Sep 8, 2026
8 checks passed
@luisleo526
luisleo526 deleted the live/abi-v4 branch September 8, 2026 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant