Skip to content

Commit f3bf9f0

Browse files
luisleo526claude
andauthored
Live runner: refuse compiled strategies a close-only stream cannot honour (capability receipt v1) (#332)
* Require compiled execution capabilities for close-only admission Add an optional versioned capability receipt and conservative pre-ledger admission. Record compiled execution declarations and analyzed request/feed requirements, refuse unsupported or unresolved requirements by name, and bind receipt bytes before the routing identity wrap. Cover generated-library refusals, receipt protocol and determinism, legacy warning compatibility, changed-receipt recovery and admitted-class batch/stream equivalence. Document display-only builtin refusal and the unchanged conservative request and POOC policy. This additive extension belongs to the next release. Record it under Unreleased without changing the released notes, base C ABI version, kernel, matching, margin or default batch computation. * Expect the capability warning in the routing E2E A legacy example library without a capability receipt now prints the documented warning at startup; the routing test parses the remaining stderr lines as delivery-error JSON, so it must skip that line like the checked settings warning. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
1 parent 8640bca commit f3bf9f0

76 files changed

Lines changed: 13133 additions & 32 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,35 @@ README's *Releases* section and on the GitHub releases page. From 1.0.0 the
55
version number follows semantic versioning over the surfaces the
66
[public contract](docs/pages/public-contract.md) lists.
77

8+
## Unreleased
9+
10+
### Compiled execution capabilities and close-only admission
11+
12+
- Add the optional, versioned generated-strategy C ABI extension
13+
`PF_CAPABILITIES_API_VERSION`, `strategy_capabilities_api_version()` and
14+
`strategy_capabilities_receipt()`. The immutable canonical JSON receipt records
15+
compiled execution declarations and analyzed request/feed requirements, using
16+
the checked-settings caller-buffer protocol. It proves declarations, not
17+
general batch-versus-stream equivalence; the base C ABI version is unchanged.
18+
- Before binding its ledger, `pineforge-live` refuses receipt-carrying libraries
19+
whose requirements its close-only stream cannot honor: intrabar calculation
20+
(`calc_on_every_tick`, `calc_on_order_fills`, `calc_on_every_history_tick`),
21+
`process_orders_on_close`, bar magnification, standard-OHLC fills, nonzero
22+
limit-fill verification, declaration-owned timeframes, FX conversion,
23+
auxiliary/native/recorded request feeds, historical probe overrides, `varip`,
24+
and unresolved declarations. Every `request.*` site is refused whatever its
25+
timeframe; admitting tested request or POOC shapes is a separate follow-up.
26+
End-of-history/realtime builtins (`barstate.islast`,
27+
`barstate.islastconfirmedhistory`, `last_bar_index`, `last_bar_time`,
28+
`barstate.isrealtime`, `timenow`) are also refused, including display-only use,
29+
because their endpoint or live-phase semantics are not certified batch-equivalent.
30+
- Refusals name the requirement instead of silently running a different strategy.
31+
Accepted receipt bytes are bound into deployment identity and recovery. Legacy
32+
libraries without the extension warn that eligibility cannot be proved and
33+
continue, including their requests; the warning is not an eligibility proof.
34+
Regenerate and relink with the paired next-release engine/codegen to obtain the
35+
receipt. Default batch computation, matching and margin are unchanged.
36+
837
## 1.1.0 — 2026-10-04
938

1039
A minor release: it adds to the C ABI. `<pineforge/pineforge.h>` declares six

‎README.md‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ The separate PineForge compiler, [`pineforge-codegen`](https://github.com/pinefo
3333
- **Open runtime.** The engine and native live runner are Apache-2.0. The separately distributed [PineForge compiler](https://github.com/pineforge-4pass/pineforge-codegen-oss/blob/main/LICENSE) uses PolyForm Noncommercial terms with additional personal-trading permission; commercial use requires a separate license. Public reference strategies, benchmarks and validation tooling are available in their respective repositories; the community-script test set is not redistributed.
3434
- **Fast.** In-process, no interpreter: median **36× faster than PyneCore** on the 200 strategies both engines time, measured at engine `35db01c8` on 2026-09-29 on an AWS c7a.8xlarge with PyneCore timed as a subprocess, its interpreter start-up included (PineForge runs a median 807k bars/s per strategy over its 201 slots with the bar magnifier on; [method](benchmarks/results/speed.md)). Parameter sweeps re-run a loaded `.so` with new inputs — no recompile, no fork.
3535
- **Deterministic to the bit.** Two runs with the same inputs produce identical trade lists. Same on Linux and macOS.
36-
- **Yours to embed.** 120 `PF_API` declarations across two headers — `pineforge.h`'s 77 (62 runtime exports and up to 15 per-strategy functions a generated module defines) and `native_c_api.h`'s 43 native-host declarations — an ABI that is append-only within a major version from 1.0 on. Call it from C, Python, Rust, Go, Node, Julia — or let an AI agent drive it over MCP.
36+
- **Yours to embed.** 122 `PF_API` declarations across two headers — `pineforge.h`'s 79 (62 runtime exports and up to 17 per-strategy functions a generated module defines) and `native_c_api.h`'s 43 native-host declarations — an ABI that is append-only within a major version from 1.0 on. Call it from C, Python, Rust, Go, Node, Julia — or let an AI agent drive it over MCP.
3737

3838
---
3939

@@ -482,18 +482,21 @@ API, the script ABI epoch and the pairing with codegen — is the
482482
[public contract](docs/pages/public-contract.md); what a 0.x user must act on
483483
is in [CHANGELOG.md](CHANGELOG.md).
484484

485-
A newly generated strategy `.so` exposes 76 compiled-strategy `PF_API` declarations
486-
(62 runtime implementations plus fourteen generated exports) plus 43 native-host
487-
declarations: 119 `PF_API` exports in total; a script that declares
488-
`use_bar_magnifier = true` also exports `strategy_declares_bar_magnifier`, the
489-
77th declaration of `pineforge.h`. Older modules lack the six opt-in
490-
[checked-settings exports](docs/checked-settings.md). In an optimized build `nm -gU` also shows libc++'s
485+
A newly generated strategy `.so` exposes 78 compiled-strategy `PF_API` declarations
486+
(62 runtime implementations plus sixteen generated exports) plus 43 native-host
487+
declarations: 121 `PF_API` exports in total; a script that declares
488+
`use_bar_magnifier = true` also exports `strategy_declares_bar_magnifier`, bringing
489+
the inventories to 79 and 122. Older modules lack the six opt-in
490+
[checked-settings exports](docs/checked-settings.md). Older modules can also lack
491+
the two opt-in [execution-capability exports](docs/strategy-capabilities.md).
492+
In an optimized build `nm -gU` also shows libc++'s
491493
`std::piecewise_construct`; no project-internal C++ symbol is exported. The two
492494
inventories are pinned by `scripts/check_c_abi_runtime.py`:
493495

494496
| Symbol | Role |
495497
|---|---|
496498
| `strategy_create` / `strategy_free` | Allocate / release a strategy instance |
499+
| `strategy_capabilities_api_version` / `strategy_capabilities_receipt` | Optional immutable compiled execution requirements for stream admission |
497500
| `run_backtest` / `run_backtest_full` | Run with auto-detected timeframe / with timeframe + magnifier configuration |
498501
| `report_free` | Free arrays inside a filled `pf_report_t` |
499502
| `strategy_closed_trade_entry_incarnation` | Per-run physical entry provenance of a closed trade |

‎docs/checked-settings.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
# Checked generated-strategy settings
22

33
This is an additive, opt-in extension to the generated strategy C ABI, not a
4-
kernel execution change or a native capability receipt. Regenerate and relink
4+
kernel execution change or a compiled capability receipt; the separate
5+
[execution-capabilities extension](strategy-capabilities.md) supplies that proof. Regenerate and relink
56
the strategy with the paired codegen and engine to obtain it. The legacy
67
factory, setters and batch calls keep their successful-call behavior and
78
default computations; all generated C boundaries now contain C++ exceptions.

‎docs/coverage.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -62,11 +62,12 @@
6262
## Public C ABI
6363

6464
`<pineforge/pineforge.h>` is the **single canonical consumer header**. It has
65-
exactly 77 public `PF_API` declarations: 62 runtime implementations and fifteen
65+
exactly 79 public `PF_API` declarations: 62 runtime implementations and seventeen
6666
per-strategy generated exports. Every compiled PineForge strategy `.so` exports
6767
that public set, except that `strategy_declares_bar_magnifier` is exported only
6868
by a script that declares `use_bar_magnifier = true`; older modules lack the six
69-
opt-in [checked-settings exports](checked-settings.md). The historical 28-symbol module sentence was not a current
69+
opt-in [checked-settings exports](checked-settings.md) and the two opt-in
70+
[execution-capability exports](strategy-capabilities.md). The historical 28-symbol module sentence was not a current
7071
module inventory; the grouped table below is a guide, not the count:
7172

7273

@@ -131,7 +132,7 @@ single `.hpp`):
131132

132133
| Module | Header | Source | Pine-facing role |
133134
| ------------------ | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
134-
| Public C ABI | `pineforge.h` | `c_abi.cpp` (+ layout `static_assert`s) | 77 public `PF_API` declarations: 62 runtime implementations plus fifteen per-strategy generated exports. `strategy_configure_native_fx_curve_v1` stages the additive native FX curve. |
135+
| Public C ABI | `pineforge.h` | `c_abi.cpp` (+ layout `static_assert`s) | 79 public `PF_API` declarations: 62 runtime implementations plus seventeen per-strategy generated exports. `strategy_configure_native_fx_curve_v1` stages the additive native FX curve. |
135136
| Engine | `engine.hpp` | `engine_run.cpp`, `engine_stream.cpp`, `engine_execution.cpp`, `engine_orders.cpp`, `engine_path_resolve.cpp`, `engine_trade_accessors.cpp`, `engine_security.cpp`, `engine_lower_tf.cpp`, `engine_report.cpp`, `native_execution_consumer.cpp` | One-shot and continuous lifecycle, native request matching/settlement, orders, reports, inputs / syminfo, magnifier, TF aggregation, and `request.security` plumbing. |
136137
| Engine internals | `engine_internal.hpp` | (private cross-TU header) | `pineforge::internal::*` types and helpers shared between engine `.cpp` partitions; not part of the public ABI. |
137138
| Technical analysis | `ta.hpp` | `ta_moving_averages.cpp`, `ta_oscillators.cpp`, `ta_volatility_trend.cpp`, `ta_extremes_volume.cpp`, `ta_misc.cpp` | Official `ta.*` functions and series variables backed by stateful runtime classes with `compute` / `recompute`, plus `pivot_point_levels(...)` free function. |

‎docs/design/native-feature-parity.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,7 @@ Columns: **Feature** (adapter mechanism, cited) · **Native today** (`yes` / `pa
121121
| CT4 | Partial-bar view / open-bar lookahead guard | **no** — the open hook receives the *complete* script bar: `invoke_bar_open_callback(engine, bar, point)` native_execution_consumer.cpp:7190 with `engine.current_bar_ = open_view` native_execution_consumer.cpp:7241 (the complete bar unless `OpenOnly`) **Closed:** `current_partial_bar()` native_host.hpp:1073 is the lookahead-free bar so far, through the last path point consumed (a fill inside a segment reads it at the segment's origin), and `NativeOpenBarView::OpenOnly` native_run_spec.hpp:109 masks `on_native_bar_open`'s bar to `H = L = C = open` as presentation only — the complete bar is restored before the open match (`tests/test_native_calc_timing.cpp`). | K | L5 | F:E3 | (single-source) The adapter needs the full bar, so the guard is opt-in. |
122122
| CT5 | Raw input observation before aggregation | **yes** — `on_native_input` native_host.hpp:875, `NativeInputContext` native_host.hpp:773-778 | K | — | O:F3 | (single-source row) |
123123
| CT6 | Calculate on order fills (scheduler `PineScheduler::recalculate` pine_scheduler_native.cpp:669, `coof_recalculation_due` pine_scheduler_native.cpp:635, the first-open chain `kFirstOpenLoopGuard` pine_scheduler_native.cpp:790; `begin_coof_recalc` pine_adapter.cpp:6209, `flush_coof_tail` pine_adapter.cpp:19930, `end_coof_recalc` pine_adapter.cpp:6251) | **partial** — `on_native_applied` native_host.hpp:938 fires mid-path after each fill native_execution_consumer.cpp:7096, native_execution_consumer.cpp:7069-7099; commands are legal there `commands_allowed` native_execution_consumer.cpp:1419-1427; a request born there is eligible on the unconsumed rest of the bar `born_on_remaining_path` native_execution_consumer.cpp:5763-5797, `remaining_path_delivery` native_order.cpp:2061-2069. Missing: calculation re-entry, a documented chronology, a bar-so-far view **Closed:** `NativeCalculationTrigger::BarCloseAndFills` native_run_spec.hpp:96 drives one `OrderFill` recalculation per applied execution at that event's own cursor, delivered as `on_native_recalculate(..., OrderFill, cause)` and bounded by `max_recalculations_per_point` (`tests/test_native_calc_timing.cpp`). | K+A | L5 | F:E4 O:F5 S:C2 | **Disagree:** F partial / O, S no; F, S v1 / O deferrable. R5-5: v1, sub-bar hook + chronology contract, adapter path unchanged. TV parts → CT11. |
124-
| CT7 | Calculate on every tick | **partial** — `on_native_tick` native_host.hpp:878 fires per accepted realtime print native_execution_consumer.cpp:7329-7333; no per-tick calculation in batch; the guide pinned "close-only" (`docs/pages/native-engine.md`, "Batch OHLCV vs ticks vs quiet") **Closed:** `NativeCalculationTrigger::EveryModeledPoint` native_run_spec.hpp:97 adds one `Tick` recalculation at every modeled point of the delivered path and at every observed print, in batch and in a stream alike (`tests/test_native_calc_timing.cpp`). | K | L5 | F:E5 O:F4 S:C3 | **Disagree:** F yes / O partial / S no. R5-5: v1. Not an adapter feature either: absent from `PineStrategyConfig` pine_adapter.hpp:62-76, the runner rejects it (`calc_on_every_tick` main.cpp:216), the Pine stream route refuses `calc_on_order_fills` pine_strategy_host.cpp:214-216. |
124+
| CT7 | Calculate on every tick | **partial** — `on_native_tick` native_host.hpp:878 fires per accepted realtime print native_execution_consumer.cpp:7329-7333; no per-tick calculation in batch; the guide pinned "close-only" (`docs/pages/native-engine.md`, "Batch OHLCV vs ticks vs quiet") **Closed:** `NativeCalculationTrigger::EveryModeledPoint` native_run_spec.hpp:97 adds one `Tick` recalculation at every modeled point of the delivered path and at every observed print, in batch and in a stream alike (`tests/test_native_calc_timing.cpp`). | K | L5 | F:E5 O:F4 S:C3 | **Disagree:** F yes / O partial / S no. R5-5: v1. Not an adapter feature either: absent from `PineStrategyConfig` pine_adapter.hpp:62-76, the runner rejects it (`calc_on_every_tick` main.cpp:218), the Pine stream route refuses `calc_on_order_fills` pine_strategy_host.cpp:214-216. |
125125
| CT8 | Historical intrabar calculation (per lower-TF sub-bar / magnifier sample) | **no** — with an `IntrabarPath` the callback still fires once per script bar native_execution_consumer.cpp:7764-7906 **Closed:** `EveryModeledPoint` recalculates at each intrabar sample, and `on_native_sub_bar` native_host.hpp:928 fires once after each retained lower-timeframe sub-bar's whole matching path (`tests/test_native_calc_timing.cpp`). | K | L5 | F:E6 O:F8 S:C4 | — |
126126
| CT9 | Intrabar (magnifier) matching path + path-order policy (`volume_weighted_cap` pine_adapter.cpp:2559-2615) | **yes** — `IntrabarPath` native_run_spec.hpp:362-403, delivery `deliver_confirmed_script` native_execution_consumer.cpp:7549-7815; `NativePathOrder` native_run_spec.hpp:335-339 | K+A | — | F:E7 F:G5 O:F7 S:C4 | **Disagree:** S partial (it wants callback cadence → CT8). TV sampler choice is A: TradingView's own intrabars at its table's timeframe, each owned by the chart bar of its last minute, built from the finer feed (`tradingview_magnifier_bars` magnifier_intrabars.hpp:44, R5 lane MAG-INTRABAR). |
127127
| CT10 | Batch → warmup → realtime forward execution | **yes** — `stream_begin` engine.hpp:1771, `stream_push_bar` engine.hpp:1777, `stream_end` engine.hpp:1781, `NativeRunPhase` native_host.hpp:41, C ABI `strategy_stream_begin` c_abi.cpp:613 | K+A | — | F:E8 O:J3 S:C5 | **Disagree:** S partial. R5-1: the lifecycle is native; Pine trade-start suppression and realtime-tail / probe rules (`trading_window_active` pine_strategy_commands.cpp:15-21, `prepare_native_begin` pine_strategy_host.cpp:153-239) are A. S's "neutral tail + trade-window policy" is single-source and unassigned (§4 Q8). |
@@ -301,7 +301,7 @@ Depends on L7 (working view), L3 (so `Sized` is in v1 of the C request) and the
301301
302302
- Move `runner/examples/native_market_strategy.cpp` and `native_selected_strategy.cpp` to a top-level `examples/native/`, built by `PINEFORGE_BUILD_EXAMPLES` (then "none yet" and guarding nothing CMakeLists.txt:37; the examples build only under `PINEFORGE_BUILD_LIVE_RUNNER` CMakeLists.txt:465-466, with `native_market_example` runner/CMakeLists.txt:54). Link the kernel target (S), or `PineForge::pineforge` during migration (O) — no SQLite / curl / OpenSSL (runner/CMakeLists.txt:1-4).
303303
- Build (1) standalone executables with a `main()` that runs batch + stream on embedded bars (the guide already contains that `main` native-engine.md:3237-3305) and (2) the same MODULE targets the runner tests load; keep the C-ABI shims so the live runner can still `dlopen` them (F, O, S).
304-
- Update the three places that pin the paths: `native_market_example` runner/CMakeLists.txt:54, `NATIVE_EXAMPLES` check_native_include_independence.py:49, the tests at `test_native_example_batch` runner/CMakeLists.txt:122.
304+
- Update the three places that pin the paths: `native_market_example` runner/CMakeLists.txt:54, `NATIVE_EXAMPLES` check_native_include_independence.py:49, the tests at `test_native_example_batch` runner/CMakeLists.txt:126.
305305
- Add a minimal "hello, kernel" host (~60 lines, no C ABI — O) and one example per v1 feature lane as it lands; each doubles as that lane's twin host (F).
306306
- Add `include/pineforge/native_module.hpp` with `PINEFORGE_EXPORT_NATIVE_STRATEGY(Class)` to replace the ~70 hand-written `extern "C"` lines per example (F, single-source).
307307
- Ship the native API reference and the standard dual-run harness of §3.1 b (S lane J).

‎docs/pages/index.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -135,19 +135,20 @@ directly, so a Pine-layer symbol reaching one is a link error.
135135

136136
## API at a glance
137137

138-
The public C surface is **120 `PF_API` declarations** across two headers:
138+
The public C surface is **122 `PF_API` declarations** across two headers:
139139

140-
- `<pineforge/pineforge.h>` — **77**: 62 runtime implementations plus fifteen
140+
- `<pineforge/pineforge.h>` — **79**: 62 runtime implementations plus seventeen
141141
per-strategy generated exports (`strategy_declares_bar_magnifier` only in a
142142
script that declares `use_bar_magnifier = true`). This is what a compiled
143143
strategy `.so` exports and what a harness calls. Older modules lack the six
144-
opt-in checked-settings exports (repository guide: `docs/checked-settings.md`).
144+
opt-in checked-settings exports (repository guide: `docs/checked-settings.md`)
145+
and two execution-capability exports (`docs/strategy-capabilities.md`).
145146
- `<pineforge/native_c_api.h>` (included by `pineforge.h`) — **43**: the other
146147
direction, where the host drives the kernel itself. Submit, replace, cancel,
147148
execute, read the book, read the lots. Additive: no symbol, struct or
148149
behaviour of the first set changes.
149150

150-
`scripts/check_c_abi_runtime.py` pins both symbol inventories (77 and 43);
151+
`scripts/check_c_abi_runtime.py` pins both symbol inventories (79 and 43);
151152
`scripts/check_native_c_api_surface.py` separately pins the host COVERAGE block.
152153

153154
| Group | Symbols | Reference |

0 commit comments

Comments
 (0)