Skip to content

Commit b7ed0ee

Browse files
luisleo526claude
andauthored
Live runner: admit proven confirmed-bar request shapes and standalone varip (#342)
* Admit proven confirmed-bar requests and closing order shapes Replace the request and process_orders_on_close refusals documented by f3bf9f0 with the confirmed-bar proof allowlist. Preserve the old receipt policy for libraries without the new versioned extension. * Prove entry-bound closing orders and reject mixed brackets * Keep unproven five-minute closing orders refused Replace the five-minute POOC admission documented by c03e06b0: its reports match batch, but its physical action timestamps do not. Add a nonvacuous same-clock request control instead of crossing a close with itself, and retain named POOC refusal before durable startup. * Match exact confirmed-bar proofs and pin refusal boundaries Address the independent review, including public terminology introduced by f3bf9f0, without changing source-layer trading code. Narrow the POOC admission sentence to exact proven order-family sets. Explicitly withdraw: POOC admits market, stop and limit entries, market closes and short stop/limit brackets on script clock `1` only. * Pin refusal of mixed-direction market brackets * Exercise named refusals for dynamic order identifiers * Pin the exact sizing profiles of closing-order proofs Narrow the settings description; omitted codegen settings are not Pine fixed-quantity defaults. Explicitly withdraw: POOC settings are limited to the default sizing/slippage/account profile, plus the separately proven market-only 100%-equity/15-tick-slippage profile. * Supply required chart context in receipt-version tests * Keep closing-order strategies refused Withdraw the closing-order admission text from c968245, 563ba41, 6a01de7 and 58fab97. Keep requests and standalone varip admitted; preserve closing-order rows only as future evidence. Withdraw these published statements explicitly so this narrowing remains named after future restacks: - An unknown optional confirmed-bar version is ignored: ordinary strategies retain their original eligibility, while requests, POOC and varip still require a recognized proof. - Explicit entry quantities, alert/comment arguments and exit OCA names are not modeled. - Market closes have long and short close-all and entry-bound proofs. - Only the proven table and order shapes admitted - POOC admits only these sorted order-family sets on script clock `1`: - POOC settings match literal proof profiles, without inferring Pine defaults: market, stop, limit and market-close sets require omitted account/sizing/slippage settings. - Partial profiles, extra otherwise-default settings and order-affecting runtime overrides are refused. - Short brackets require exactly fixed quantity 1 and explicit percent commission 0. - Stop-limit/OCA entries, trailing/relative/partial exits, long brackets, `strategy.order`, risk rules, cancellation, immediate closes and any call or argument outside the modeled allowlist remain refused by name. - The five-minute both-sided-stop report is batch-equivalent, but its physical action timestamps are not; that clock stays refused. - The separate market-only profile requires exactly explicit 100%-equity sizing and 15-tick slippage. - The sole request-plus-POOC proof is SMA(close,4) at `15` with market entries; other cross-category compositions are refused rather than inferred. - Two requests together, request-plus-varip and POOC-plus-varip are refused. - `[entry:market]`, `[entry:stop]`, `[entry:limit]`, `[entry:market, exit:short_bracket]`, and `[close:market, entry:market]`. * Shard confirmed refusal tests and allow sanitizer headroom * Mark the confirmed-bar extension as unreleased after 1.2.0 1.2.0 shipped the capability extension and its receipt-based refusals, but not the confirmed-bar receipt or its request and varip admission; the documentation said both were available since 1.2.0. This rewrites the availability sentence "This applies to both the capability extension and the runner's receipt-based admission policy." introduced by f3bf9f0, so that it covers only what 1.2.0 shipped. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
1 parent c714984 commit b7ed0ee

195 files changed

Lines changed: 45505 additions & 57 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -222,6 +222,10 @@ the [native runner guide](runner/README.md) for feed modes, symbol metadata,
222222
feed format, recovery and execution limitations. The existing validation
223223
scoreboard below describes batch backtests; it does not certify new native
224224
live behavior or real broker fills.
225+
Receipt-based confirmed-bar admission (unreleased; after 1.2.0) is limited to the
226+
[proven same-chart requests and standalone close-only varip](docs/strategy-capabilities.md#confirmed-bar-extension).
227+
`process_orders_on_close=true` remains refused for every order shape and
228+
composition, before any ledger exists.
225229
Known issue (v1.0.0, v1.0.1): a stream whose input and script timeframes are
226230
equal serves stale `request.security` values after the first realtime bar.
227231
Fixed in 1.1.0 (#325). See
@@ -497,6 +501,7 @@ inventories are pinned by `scripts/check_c_abi_runtime.py`:
497501
|---|---|
498502
| `strategy_create` / `strategy_free` | Allocate / release a strategy instance |
499503
| `strategy_capabilities_api_version` / `strategy_capabilities_receipt` | Optional immutable compiled execution requirements for stream admission |
504+
| `strategy_confirmed_bar_api_version` / `strategy_confirmed_bar_receipt` | Optional versioned metadata for proven confirmed-bar admission (after 1.2.0); legacy capability exports remain unchanged |
500505
| `run_backtest` / `run_backtest_full` | Run with auto-detected timeframe / with timeframe + magnifier configuration |
501506
| `report_free` | Free arrays inside a filled `pf_report_t` |
502507
| `strategy_closed_trade_entry_incarnation` | Per-run physical entry provenance of a closed trade |

‎docs/strategy-capabilities.md‎

Lines changed: 81 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Compiled strategy execution capabilities
22

3-
Availability: **since 1.2.0**. This applies to both the capability
4-
extension and the runner's receipt-based admission policy.
3+
Availability: **since 1.2.0** for the capability extension and the runner's
4+
receipt-based refusals; the confirmed-bar extension below is unreleased (after 1.2.0).
55

66
An additive generated-strategy C ABI extension proves compiled declarations
77
only, not general batch-versus-stream equivalence. A consumer applies its
@@ -81,9 +81,81 @@ intrabar persistence. Native feeds and historical probe/tail data are not
8181
installed by generated scripts: those requirements default false, while
8282
separately staged runtime data is still validated at stream begin.
8383

84-
## Close-only runner policy
84+
## Confirmed-bar extension
8585

86-
The runner reads and validates version 1 before execution begins or the ledger
86+
Unreleased (after 1.2.0): new generated libraries additionally export `strategy_confirmed_bar_api_version`
87+
(returning 1) and `strategy_confirmed_bar_receipt`, with the same signature and
88+
buffer/error protocol as the original receipt. The original version-1 schema
89+
and exports remain available, without new keys: older runners ignore the new
90+
symbols rather than rejecting unfamiliar fields. No engine header or runtime
91+
ABI export is needed for this optional, dynamically discovered adapter receipt.
92+
93+
Its canonical JSON has `version: 1`, `requests` (the original request fields plus
94+
the lowered `expression`), `orders` (sorted distinct classified order shapes),
95+
and `intrabar_persistence`. Feed/Heikin-Ashi flags and static clocks come from
96+
the emitter's registration decisions, not symbol-text inference. Inputs or
97+
mutable clocks remain unresolved. The runner checks agreement between both
98+
receipts and fails closed on malformed fields or missing proofs. An unknown optional
99+
confirmed-bar version is ignored: ordinary strategies retain their original
100+
eligibility, while requests and varip still require a recognized proof.
101+
POOC remains refused regardless of the optional receipt.
102+
103+
New admission requires confirmed one-minute bars, UTC/24x7 source settings,
104+
no external native configuration, and the following exact proof population:
105+
106+
| Same-chart requested series | Request clock | Script clock | Merge policy |
107+
| --- | --- | --- | --- |
108+
| `close` | `5` | `1`, `5` | lookahead off, gaps off |
109+
| `close` | `60` | `1`, `15` | lookahead off, gaps off |
110+
| `close` | `D` | `1` | lookahead off, gaps off |
111+
| `ta.sma(close, 4)` | `15` | `1` | lookahead off, gaps off |
112+
| `ta.ema(close, 3)` | `60` | `1` | lookahead off, gaps on |
113+
| `close[1]` | `timeframe.period` | `1` | lookahead off, gaps off |
114+
| Heikin-Ashi `close` | `5` | `1` | lookahead off, gaps off |
115+
116+
The clocks form an allowlist, not a monthly-clock denylist. Constants and a
117+
single-literal-call helper resolve to these same clocks. A Heikin-Ashi alias
118+
keeps the registration's transformation flag. Symbol `""` currently lowers
119+
through a foreign feed and is therefore refused, not mislabeled as chart data.
120+
121+
`process_orders_on_close=true` remains refused by name for every order shape,
122+
including market-only entries, priced entries, short brackets, market closes
123+
and request-plus-POOC compositions. CLI overrides cannot erase that compiled
124+
declaration. Nullable price legs, brackets issued with their entry, and empty
125+
entry identifiers need a systematic proof before any POOC admission.
126+
The exact order-family-set matcher and POOC batch/stream rows remain in-tree
127+
as evidence for future admission, not admitted shapes. Their runner E2Es
128+
assert refusal before any ledger, control directory or status file exists.
129+
The receipt's modeled call/argument allowlist remains metadata, not a runtime
130+
certificate for nullable prices or pending/global exits.
131+
Two requests together and request-plus-varip are also refused.
132+
Close-only `varip` is admitted on script clock `1`, never observed ticks.
133+
Every admitted source has a generated C++ batch/stream equivalence row and a
134+
runner E2E comparing physical actions and every ABI report field bitwise on a
135+
tape, at warmup splits 30, 33 and 500 and after replay. Daily requests also
136+
exercise split 1500 after the first daily boundary. The original #325 numerical pins
137+
remain unchanged. See `tests/fixtures/confirmed_capabilities/README.md`.
138+
139+
Upgrading an original-receipt runner to one that recognizes the confirmed-bar
140+
extension changes the deployment identity of every dual-receipt library,
141+
including ordinary strategies. Its old ledger cannot silently resume: keep the
142+
original runner/library for that ledger, or create a new deployment and ledger.
143+
144+
| Runner | Library | Admission |
145+
| --- | --- | --- |
146+
| Old | Original receipt or new dual receipts | Original policy; extra symbols ignored |
147+
| New | Original receipt only | Original request/POOC/varip refusals preserved |
148+
| New | Both version-1 receipts | Only the proven request table and standalone close-only varip admitted; POOC refused |
149+
| Either | No capability receipt | Existing warning and legacy behavior retained |
150+
151+
The additional receipt is hashed as `SHA256(previous_identity +
152+
":confirmed-bars-v1:" + receipt)` after the original capability receipt and
153+
before routing wrapping; its summary field is `confirmed_bar_capabilities`.
154+
Missing extra metadata leaves the old identity unchanged.
155+
156+
## Original-receipt close-only policy
157+
158+
Without the additional receipt, the runner reads and validates version 1 before execution begins or the ledger
87159
exists, independently of the settings receipt. It refuses true every-tick,
88160
order-fill, historical-tick, process-orders-on-close, magnifier and standard-OHLC-fill declarations; nonzero limit
89161
verification; non-default currency or declaration-owned timeframe; any true
@@ -97,9 +169,9 @@ runner admission policy even after the confirmed-bar stream fixes: same-chart
97169
close/SMA/EMA (including `gaps_on`), and higher-timeframe Heikin-Ashi chart-symbol
98170
requests now match batch in the generated stream equivalence tests. This does
99171
not prove arbitrary requests, foreign/auxiliary feeds, lower-timeframe arrays or
100-
historical look-ahead projections. Every request from a receipt-carrying library
101-
and `process_orders_on_close=true` remain refused; relaxing admission is a
102-
separate follow-up, not an automatic consequence of those runtime fixes.
172+
historical look-ahead projections. Every request from a library carrying only
173+
the original receipt and `process_orders_on_close=true` remain refused. New
174+
admission requires the additional receipt and the proof table above.
103175
Use of `barstate.isrealtime` and `timenow` is conservatively refused by name:
104176
codegen currently emits historical-only `false` and the current bar timestamp,
105177
respectively, in both warmup and realtime, not Pine's live phase/wall clock.
@@ -136,8 +208,8 @@ The confirmed-bar stream fixes close the earlier POOC priced-entry and
136208
default-sized both-sided-stop discrepancies (the historical audit measured
137209
batch 276 versus stream 277 trades), while the request tests establish the
138210
bounded shapes described above. This receipt still proves declarations only,
139-
not arbitrary order-shape equivalence, and keeps its conservative POOC/request
140-
refusals. See [Backtest vs live](pages/streaming.md#backtest_vs_live) for deliberate
211+
not arbitrary order-shape equivalence. Without the additional proof receipt it
212+
keeps its conservative POOC/request refusals. See [Backtest vs live](pages/streaming.md#backtest_vs_live) for deliberate
141213
historical look-ahead differences and the remaining deferred calendar-boundary
142214
limitation. Capability checks do not change default batch computation, matching
143215
or margin.

‎runner/CMakeLists.txt‎

Lines changed: 116 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ if(PINEFORGE_BUILD_TESTS)
120120
add_test(NAME native_live_report_e2e COMMAND ${Python3_EXECUTABLE}
121121
${PROJECT_SOURCE_DIR}/tests/native_live_report_e2e.py $<TARGET_FILE:pineforge-live>
122122
$<TARGET_FILE:native_live_report_oracle> ${_pf_report_strategies})
123-
set_tests_properties(native_live_report_e2e PROPERTIES TIMEOUT 120)
123+
set_tests_properties(native_live_report_e2e PROPERTIES TIMEOUT 300)
124124
add_test(NAME native_live_e2e COMMAND ${Python3_EXECUTABLE}
125125
${PROJECT_SOURCE_DIR}/tests/native_live_e2e.py $<TARGET_FILE:pineforge-live>
126126
$<TARGET_FILE:native_live_example>)
@@ -254,4 +254,119 @@ if(PINEFORGE_BUILD_TESTS AND PINEFORGE_BUILD_SOURCE_LAYER)
254254
add_test(NAME capabilities_equivalence_${capabilities_kind}
255255
COMMAND test_capabilities_equivalence_${capabilities_kind})
256256
endforeach()
257+
if(PINEFORGE_BUILD_SOURCE_LAYER)
258+
add_executable(test_confirmed_capabilities ${PROJECT_SOURCE_DIR}/tests/test_confirmed_capabilities.cpp)
259+
target_include_directories(test_confirmed_capabilities PRIVATE ${CMAKE_CURRENT_SOURCE_DIR})
260+
target_compile_features(test_confirmed_capabilities PRIVATE cxx_std_17)
261+
target_compile_options(test_confirmed_capabilities PRIVATE -UNDEBUG)
262+
add_test(NAME confirmed_capabilities_policy COMMAND test_confirmed_capabilities)
263+
set(_pf_confirmed_cases htf5_close htf5_close_active htf15_sma htf15_sma_pooc htf60_close daily_close heikinashi5
264+
htf60_ema_gaps period_previous pooc_dual_stop pooc_limit_reversal pooc_slipped_short
265+
pooc_short_exit constant_daily helper5 heikinashi_alias5 varip pooc_market pooc_close pooc_close_entry
266+
pooc_close_short pooc_close_short_entry)
267+
file(GLOB _pf_confirmed_refusal_sources CONFIGURE_DEPENDS
268+
"${PROJECT_SOURCE_DIR}/tests/fixtures/confirmed_capabilities/generated/refused_*.cpp")
269+
set(_pf_confirmed_refusals)
270+
foreach(refusal_source IN LISTS _pf_confirmed_refusal_sources)
271+
get_filename_component(refusal_case "${refusal_source}" NAME_WE)
272+
list(APPEND _pf_confirmed_refusals "${refusal_case}")
273+
endforeach()
274+
set(_pf_confirmed_libraries)
275+
foreach(confirmed_case IN LISTS _pf_confirmed_cases _pf_confirmed_refusals)
276+
add_library(confirmed_${confirmed_case} MODULE
277+
${PROJECT_SOURCE_DIR}/tests/fixtures/confirmed_capabilities/generated/${confirmed_case}.cpp
278+
$<TARGET_OBJECTS:native_live_equivalence_observer>)
279+
target_link_libraries(confirmed_${confirmed_case} PRIVATE PineForge::pineforge)
280+
target_compile_options(confirmed_${confirmed_case} PRIVATE -ffp-contract=off)
281+
if(APPLE)
282+
target_link_options(confirmed_${confirmed_case} PRIVATE "LINKER:-u,_pf_abi_version")
283+
else()
284+
target_link_options(confirmed_${confirmed_case} PRIVATE "LINKER:-u,pf_abi_version")
285+
endif()
286+
set_target_properties(confirmed_${confirmed_case} PROPERTIES PREFIX "")
287+
list(APPEND _pf_confirmed_libraries "${confirmed_case}=$<TARGET_FILE:confirmed_${confirmed_case}>")
288+
endforeach()
289+
set(_pf_confirmed_refusal_requests)
290+
set(_pf_confirmed_refusal_pooc)
291+
foreach(confirmed_library IN LISTS _pf_confirmed_libraries)
292+
if(confirmed_library MATCHES "^(pooc_|refused_pooc_|htf15_sma_pooc=)")
293+
list(APPEND _pf_confirmed_refusal_pooc "${confirmed_library}")
294+
else()
295+
list(APPEND _pf_confirmed_refusal_requests "${confirmed_library}")
296+
endif()
297+
endforeach()
298+
set(_pf_confirmed_refusal_timeout 180)
299+
if(PINEFORGE_ENABLE_SANITIZERS OR PINEFORGE_LIVE_TSAN)
300+
set(_pf_confirmed_refusal_timeout 600)
301+
endif()
302+
foreach(refusal_family requests pooc)
303+
set(refusal_libraries ${_pf_confirmed_refusal_${refusal_family}})
304+
list(LENGTH refusal_libraries refusal_library_count)
305+
set(refusal_offset 0)
306+
set(refusal_shard 1)
307+
while(refusal_offset LESS refusal_library_count)
308+
list(SUBLIST refusal_libraries ${refusal_offset} 8 refusal_shard_libraries)
309+
set(refusal_test "confirmed_capabilities_refusals_${refusal_family}_${refusal_shard}")
310+
add_test(NAME ${refusal_test} COMMAND ${Python3_EXECUTABLE}
311+
${PROJECT_SOURCE_DIR}/tests/confirmed_capabilities_refusals.py $<TARGET_FILE:pineforge-live>
312+
${refusal_shard_libraries})
313+
set_tests_properties(${refusal_test} PROPERTIES
314+
TIMEOUT ${_pf_confirmed_refusal_timeout} LABELS confirmed_refusal_shard)
315+
math(EXPR refusal_offset "${refusal_offset} + 8")
316+
math(EXPR refusal_shard "${refusal_shard} + 1")
317+
endwhile()
318+
endforeach()
319+
foreach(receipt_variant ordinary request pooc missing)
320+
add_library(confirmed_receipt_${receipt_variant} MODULE
321+
${PROJECT_SOURCE_DIR}/tests/fixtures/confirmed_capabilities/receipt_versions.cpp)
322+
target_link_libraries(confirmed_receipt_${receipt_variant} PRIVATE PineForge::pineforge)
323+
target_compile_definitions(confirmed_receipt_${receipt_variant} PRIVATE PF_RECEIPT_${receipt_variant}=1)
324+
if(APPLE)
325+
target_link_options(confirmed_receipt_${receipt_variant} PRIVATE "LINKER:-u,_pf_abi_version")
326+
else()
327+
target_link_options(confirmed_receipt_${receipt_variant} PRIVATE "LINKER:-u,pf_abi_version")
328+
endif()
329+
set_target_properties(confirmed_receipt_${receipt_variant} PROPERTIES PREFIX "")
330+
endforeach()
331+
add_test(NAME confirmed_capabilities_receipt_versions COMMAND ${Python3_EXECUTABLE}
332+
${PROJECT_SOURCE_DIR}/tests/confirmed_receipt_versions.py $<TARGET_FILE:pineforge-live>
333+
$<TARGET_FILE:confirmed_receipt_ordinary> $<TARGET_FILE:confirmed_receipt_request>
334+
$<TARGET_FILE:confirmed_receipt_pooc> $<TARGET_FILE:confirmed_receipt_missing>)
335+
foreach(confirmed_case IN LISTS _pf_confirmed_cases)
336+
if(confirmed_case MATCHES "^pooc_" OR confirmed_case STREQUAL "htf15_sma_pooc")
337+
add_test(NAME confirmed_capabilities_${confirmed_case}_refusal_e2e COMMAND ${Python3_EXECUTABLE}
338+
${PROJECT_SOURCE_DIR}/tests/confirmed_capabilities_refusals.py $<TARGET_FILE:pineforge-live>
339+
"${confirmed_case}=$<TARGET_FILE:confirmed_${confirmed_case}>")
340+
set_tests_properties(confirmed_capabilities_${confirmed_case}_refusal_e2e PROPERTIES TIMEOUT 180)
341+
else()
342+
add_test(NAME confirmed_capabilities_${confirmed_case}_e2e COMMAND ${Python3_EXECUTABLE}
343+
${PROJECT_SOURCE_DIR}/tests/confirmed_capabilities_e2e.py $<TARGET_FILE:pineforge-live>
344+
$<TARGET_FILE:native_live_report_oracle> ${confirmed_case} $<TARGET_FILE:confirmed_${confirmed_case}>)
345+
set_tests_properties(confirmed_capabilities_${confirmed_case}_e2e PROPERTIES TIMEOUT 900)
346+
endif()
347+
add_executable(test_confirmed_equivalence_${confirmed_case}
348+
${PROJECT_SOURCE_DIR}/tests/test_stream_security_equivalence.cpp
349+
${PROJECT_SOURCE_DIR}/tests/fixtures/confirmed_capabilities/generated/${confirmed_case}.cpp)
350+
target_link_libraries(test_confirmed_equivalence_${confirmed_case} PRIVATE PineForge::pineforge)
351+
target_compile_options(test_confirmed_equivalence_${confirmed_case} PRIVATE -UNDEBUG -ffp-contract=off)
352+
target_compile_definitions(test_confirmed_equivalence_${confirmed_case} PRIVATE PINEFORGE_SECURITY_CASE="${confirmed_case}")
353+
add_test(NAME confirmed_capabilities_${confirmed_case}_equivalence COMMAND test_confirmed_equivalence_${confirmed_case} 1 500)
354+
if(confirmed_case MATCHES "^pooc_" OR confirmed_case STREQUAL "htf15_sma_pooc")
355+
set_tests_properties(confirmed_capabilities_${confirmed_case}_equivalence PROPERTIES LABELS "future_admission")
356+
endif()
357+
endforeach()
358+
foreach(confirmed_control htf5_close_active htf60_close)
359+
if(confirmed_control STREQUAL "htf60_close")
360+
set(_pf_confirmed_clock 15)
361+
else()
362+
set(_pf_confirmed_clock 5)
363+
endif()
364+
add_test(NAME confirmed_capabilities_${confirmed_control}_tf${_pf_confirmed_clock}_e2e COMMAND ${Python3_EXECUTABLE}
365+
${PROJECT_SOURCE_DIR}/tests/confirmed_capabilities_e2e.py $<TARGET_FILE:pineforge-live>
366+
$<TARGET_FILE:native_live_report_oracle> ${confirmed_control} $<TARGET_FILE:confirmed_${confirmed_control}> ${_pf_confirmed_clock})
367+
set_tests_properties(confirmed_capabilities_${confirmed_control}_tf${_pf_confirmed_clock}_e2e PROPERTIES TIMEOUT 900)
368+
add_test(NAME confirmed_capabilities_${confirmed_control}_tf${_pf_confirmed_clock}_equivalence
369+
COMMAND test_confirmed_equivalence_${confirmed_control} ${_pf_confirmed_clock} 500)
370+
endforeach()
371+
endif()
257372
endif()

0 commit comments

Comments
 (0)