Skip to content

Commit 95f4fd1

Browse files
authored
C ABI: strategy_initial_capital, so the report writer names simple_proportional_overflow in every run (#565)
## What Adds the C ABI export `double strategy_initial_capital(pf_strategy_t)`, so the report writer can name `simple_proportional_overflow` exactly, in every run. Before, `docker/run_json.py` named that reason in `diagnostics.annualization` only when it could prove the first curve mark was the initial capital (the selected-window anchor, or an ordinary run whose first mark had no open profit and no trade entered at bar 0). In every other run of one day or more whose compounded `cagr` was not finite, the reported `cagr` was the simple proportional value the kernel computed, but the reason was silently absent. The revision-3.1 note said a later release could close the gap by reading the capital from the engine; this is that release. - `strategy_initial_capital` returns the capital the configured run uses: the staged native specification before a run begins, the engine's stored capital otherwise. It returns NaN for a null handle (the same sentinel as `strategy_current_equity`) and adds no kernel state. - `docker/run_json.py` reads it once per run behind a missing-symbol guard and uses it as the initial capital when it is finite and positive. A library without the export keeps the earlier behaviour byte for byte. The subday branch, every existing absence, the overflow handling and the simple `cagr` value are unchanged. - The export appears in every list that pins the runtime symbols: the checker's names and counts (113 to 114 declarations, 93 to 94 runtime implementations), the pinned counts in the export-list test and in the docs, and the kernel residual table of the boundary ADR. The frozen legacy export list stays as it is. - The comment in `src/engine_metrics.cpp` that described the one-day-boundary ranking inversion backwards now says what the documentation says (compounded above simple); no code in that file changed. - Documentation: the report schema and the revision-3.1 page describe the exact behaviour and the retained gap for older libraries; the C ABI pages and README counts follow. ## Verification - Red first: with the export present and `docker/run_json.py` at base, the new end-to-end test `test_initial_capital_export_closes_the_unproven_overflow_gap` fails (the reason is absent); on the base tree the new C test and checker lists fail to build and check for the missing export. - A first lab run found one real defect, fixed here: right after `strategy_configure_native_v1` the first accessor returned the engine's member default, because the native route only stages the specification and copies the capital at the start of the run. The accessor now reads the staged specification; the C test asserts it for a reused host as well. - All eight lab legs (preflight, release, debug, sanitizers, kernel, docs, parity, subset) pass on the pre-merge head; release and debug pass again on the head merged with main. The edit moved nine citations of the report-sink page; they were re-pointed, and the doc-anchors stage of the preflight on the final head holds them. - The corpus parity leg and the subset leg report no change in report bytes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 parent c85a40b commit 95f4fd1

27 files changed

Lines changed: 292 additions & 136 deletions

‎README.md‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ The separate PineForge compiler, [`pineforge-codegen`](https://github.com/pinefo
3333
- **Open runtime.** The engine and native live runner are Apache-2.0. The separately distributed [PineForge compiler](https://github.com/pineforge-4pass/pineforge-codegen-oss/blob/main/LICENSE) is source-available under the PineForge Source License 1.2 from its release 1.3.0 (release 1.2.0 keeps the PineForge Source License 1.1, and releases up to 1.1.0 keep the PolyForm Noncommercial terms they shipped with); personal trading is free, and commercial use requires a separate license; the capital in an account a proprietary-trading firm or funded-trader program provides or allocates, including a challenge, evaluation or simulated account, is investment capital. Public reference strategies, benchmarks and validation tooling are available in their respective repositories; the community-script test set is not redistributed.
3434
- **Fast.** In-process, no interpreter: median **36× faster than PyneCore** on the 200 strategies both engines time (100 corpus probes and 100 held-out private strategies), measured at engine `35db01c8` on 2026-09-29 on an AWS c7a.8xlarge with PyneCore timed as a subprocess, its interpreter start-up included (PineForge runs a median 807k bars/s per strategy over its 201 slots with the bar magnifier on; [method](benchmarks/results/speed.md)). Parameter sweeps re-run a loaded `.so` with new inputs — no recompile, no fork.
3535
- **Deterministic to the bit.** Two runs with the same inputs produce identical trade lists.
36-
- **Yours to embed.** 158 `PF_API` declarations across six installed headers — `pineforge.h`'s 102 (82 of the 93 runtime exports and up to 20 per-strategy functions a generated module defines), eleven more runtime declarations in `execution_observer.h`, `selected_window.h` and `report_sink.h` (three, three and five), `native_c_api.h`'s 43 native-host declarations and `selected_window_plan.h`'s two planner declarations — an ABI that is append-only within a major version from 1.0 on. Call it from C, Python, Rust, Go, Node, Julia — or let an AI agent drive it over MCP.
36+
- **Yours to embed.** 159 `PF_API` declarations across six installed headers — `pineforge.h`'s 103 (83 of the 94 runtime exports and up to 20 per-strategy functions a generated module defines), eleven more runtime declarations in `execution_observer.h`, `selected_window.h` and `report_sink.h` (three, three and five), `native_c_api.h`'s 43 native-host declarations and `selected_window_plan.h`'s two planner declarations — an ABI that is append-only within a major version from 1.0 on. Call it from C, Python, Rust, Go, Node, Julia — or let an AI agent drive it over MCP.
3737

3838
Recorded outputs and their additional API declarations described here are
3939
part of **1.5.0**.
@@ -540,11 +540,11 @@ is in [CHANGELOG.md](CHANGELOG.md).
540540

541541
The counts below describe the current API, including recorded outputs, the report sink and scratch-table selection; recorded outputs are not part of 1.4.0.
542542

543-
A newly generated strategy `.so` exposes 109 compiled-strategy `PF_API` declarations
544-
(93 runtime implementations plus sixteen generated exports) plus 43 native-host
545-
declarations: 152 `PF_API` exports in total; `use_bar_magnifier = true` adds
543+
A newly generated strategy `.so` exposes 110 compiled-strategy `PF_API` declarations
544+
(94 runtime implementations plus sixteen generated exports) plus 43 native-host
545+
declarations: 153 `PF_API` exports in total; `use_bar_magnifier = true` adds
546546
`strategy_declares_bar_magnifier` and a module that records [outputs](docs/outputs.md) its
547-
three outputs exports, bringing the inventories to at most 113 and 156. Older modules lack the six opt-in
547+
three outputs exports, bringing the inventories to at most 114 and 157. Older modules lack the six opt-in
548548
[checked-settings exports](docs/checked-settings.md). Older modules can also lack
549549
the two opt-in [execution-capability exports](docs/strategy-capabilities.md).
550550
In an optimized build `nm -gU` also shows libc++'s
@@ -589,6 +589,7 @@ inventories are pinned by `scripts/check_c_abi_runtime.py`:
589589
| `strategy_pending_order_fill_qty` / `_level_resolved` / `_effective_levels` / `strategy_trail_best_price` | ABI v4: engine-computed values for a resting order — the quantity it would open if filled at a given price, whether its relative offsets resolve yet, its resolved stop/limit/trail-activation levels, and the live position's trail extreme |
590590
| `strategy_position_avg_price` / `strategy_position_cycle_seq` / `strategy_position_size` | ABI v4: the live position's volume-weighted average entry price, its cycle id, and its script-facing signed size |
591591
| `strategy_closed_trade_entry_id` / `_exit_id` / `_exit_comment` / `_close_cause` | ABI v4: per-closed-trade id/comment strings and a `close_cause` (`pf_close_cause_t`: `SCRIPT`, `BRACKET`, `LIQUIDATION` for a margin call, `RISK_LIMIT` for the intraday loss cap, `FILL_CAP` for the intraday fill cap, `RANGE_END`), indexed like `strategy_closed_trade_entry_incarnation` |
592+
| `strategy_initial_capital` | ABI v4: the engine's effective configured initial capital, after the native run specification or strategy override; `NaN` if `s` is `NULL` or the read raises |
592593
| `strategy_current_equity` / `strategy_script_bars_processed` | ABI v4: `initial_capital + netprofit` (not Pine's `strategy.equity`, which also adds open profit); total script bars dispatched by the most recent run |
593594

594595
### Driving the kernel from C

‎docker/run_json.py‎

Lines changed: 56 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -3979,22 +3979,26 @@ def _num(x, *, report_metric=False):
39793979
MIN_ANNUALIZATION_SPAN_MS = 86_400_000
39803980

39813981

3982-
def _annualization_diagnostics(curve, metrics, trades=(), window_anchor=False):
3982+
def _annualization_diagnostics(curve, metrics, trades=(), window_anchor=False,
3983+
initial_capital=None):
39833984
"""Name the simple proportional form of cagr, and of the calmar made from it.
39843985
39853986
No span makes cagr, calmar, sharpe_bar or sortino_bar null, so the writer
39863987
names the form from what it holds. Under one day the kernel always takes
39873988
the simple form: a finite cagr on a positive sub-day span is flagged. From
39883989
one day on it takes the compounded form unless that is not finite, so the
39893990
writer repeats the kernel's arithmetic on the curve's own marks: the ratio
3990-
of the last mark to the initial capital. The initial capital is the first
3991-
mark only when the rows prove it untouched: the selected-window anchor
3992-
(window_anchor), or no open profit and no trade entered at it. With no such
3993-
proof the writer makes no claim and names nothing. calmar is cagr over the
3994-
drawdown percent, so it carries the same flag as the cagr it was computed
3995-
from: its key is present exactly when the cagr key is and calmar is not
3996-
null (zero drawdown leaves it null: then no calmar key). The bar ratios use
3997-
one formula on every span and have no reason.
3991+
of the last mark to the engine's initial capital when the optional
3992+
strategy_initial_capital export supplied a finite positive value. Older
3993+
libraries without that export fall back to the first mark only when the
3994+
rows prove it untouched: the selected-window anchor (window_anchor), or no
3995+
open profit and no trade entered at it. An invalid exported value also uses
3996+
that fallback. With no such proof the writer makes no claim and names
3997+
nothing. calmar is cagr over the drawdown percent, so it carries the same
3998+
flag as the cagr it was computed from: its key is present exactly when the
3999+
cagr key is and calmar is not null (zero drawdown leaves it null: then no
4000+
calmar key). The bar ratios use one formula on every span and have no
4001+
reason.
39984002
Python's integer subtraction is exact across the full int64 timestamp
39994003
domain; convert the difference to double just as the kernel does.
40004004
"""
@@ -4008,10 +4012,18 @@ def _annualization_diagnostics(curve, metrics, trades=(), window_anchor=False):
40084012
if span_ms < MIN_ANNUALIZATION_SPAN_MS:
40094013
form = "simple_proportional_subday"
40104014
else:
4011-
if not window_anchor and (curve[0]["open_profit"] != 0
4012-
or any(row["entry_bar_index"] <= 0 for row in trades)):
4013-
return {}
4014-
capital, final = curve[0]["equity"], curve[-1]["equity"]
4015+
capital = initial_capital
4016+
try:
4017+
valid_capital = (
4018+
capital is not None and math.isfinite(capital) and capital > 0)
4019+
except (TypeError, ValueError, OverflowError):
4020+
valid_capital = False
4021+
if not valid_capital:
4022+
if not window_anchor and (curve[0]["open_profit"] != 0
4023+
or any(row["entry_bar_index"] <= 0 for row in trades)):
4024+
return {}
4025+
capital = curve[0]["equity"]
4026+
final = curve[-1]["equity"]
40154027
if not (math.isfinite(capital) and capital > 0 and math.isfinite(final) and final > 0):
40164028
return {}
40174029
span_years = span_ms / (365.25 * 86400.0 * 1000.0)
@@ -4027,9 +4039,11 @@ def _annualization_diagnostics(curve, metrics, trades=(), window_anchor=False):
40274039
return {"cagr": form, "calmar": form}
40284040

40294041

4030-
def _annualization_block(curve, metrics, trades=(), window_anchor=False) -> dict:
4042+
def _annualization_block(curve, metrics, trades=(), window_anchor=False,
4043+
initial_capital=None) -> dict:
40314044
"""The diagnostics entry for revision 3.1: present only when cagr used a simple form."""
4032-
reasons = _annualization_diagnostics(curve, metrics, trades, window_anchor)
4045+
reasons = _annualization_diagnostics(
4046+
curve, metrics, trades, window_anchor, initial_capital)
40334047
return {"annualization": reasons} if reasons else {}
40344048

40354049

@@ -4055,9 +4069,10 @@ def build_report_dict(report: ReportC, ohlcv_path: Path,
40554069
applied_overrides: dict[str, str],
40564070
applied_runtime: dict[str, object] | None = None,
40574071
trade_entry_incarnations=None,
4058-
window_anchor: bool = False) -> dict:
4059-
# window_anchor: the curve's first point is the selected-window anchor, the
4060-
# engine's own initial capital by construction (_annualization_diagnostics).
4072+
window_anchor: bool = False,
4073+
initial_capital=None) -> dict:
4074+
# window_anchor: the curve's first point is the selected-window anchor. It
4075+
# remains the fallback proof when the runtime lacks strategy_initial_capital.
40614076
def trade_at(i):
40624077
t = report.trades[i]
40634078
return {
@@ -4165,7 +4180,8 @@ def point_at(i):
41654180
"magnifier_sub_bars_total": int(report.magnifier_sub_bars_total),
41664181
"magnifier_sample_ticks_total": int(report.magnifier_sample_ticks_total),
41674182
"bar_magnifier_enabled": bool(report.bar_magnifier_enabled),
4168-
**_annualization_block(equity_curve, metrics, trades, window_anchor),
4183+
**_annualization_block(
4184+
equity_curve, metrics, trades, window_anchor, initial_capital),
41694185
},
41704186
"trades": trades,
41714187
"metrics": metrics,
@@ -4950,12 +4966,13 @@ def _make_state():
49504966
# list pre-pass. These read-only values are revisited during output.
49514967
for i in range(report.trades_len):
49524968
trade_entry_incarnations(i)
4969+
engine_initial_capital = _read_strategy_initial_capital(lib, state)
49534970
builder = _build_sink_report if sink is not None else build_report_dict
49544971
out = builder(
49554972
report, args.ohlcv, fed_count, plan["fed_first_data_ms"], plan["fed_last_data_ms"],
49564973
elapsed, inputs, overrides, applied_runtime,
49574974
sink if sink is not None else trade_entry_incarnations,
4958-
window_anchor=True)
4975+
window_anchor=True, initial_capital=engine_initial_capital)
49594976
if sink is None:
49604977
_validate_report_result(out)
49614978
_add_scratch_arena_diagnostics(lib, out["diagnostics"], args)
@@ -5544,11 +5561,13 @@ def _run(st, rep):
55445561
if trade_entry_incarnations is not None:
55455562
for i in range(report.trades_len):
55465563
trade_entry_incarnations(i)
5564+
engine_initial_capital = _read_strategy_initial_capital(lib, state)
55475565
builder = _build_sink_report if sink is not None else build_report_dict
55485566
out = builder(
55495567
report, args.ohlcv, n, first_ts, last_ts,
55505568
elapsed, inputs, overrides, applied_runtime,
5551-
sink if sink is not None else trade_entry_incarnations)
5569+
sink if sink is not None else trade_entry_incarnations,
5570+
initial_capital=engine_initial_capital)
55525571
if sink is None:
55535572
_validate_report_result(out)
55545573
_add_scratch_arena_diagnostics(lib, out["diagnostics"], args)
@@ -7101,14 +7120,15 @@ def _sink_summary(report, sink, *, records=None):
71017120

71027121
def _build_sink_report(report, ohlcv_path, n_bars, first_ts, last_ts, elapsed,
71037122
applied_inputs, applied_overrides, applied_runtime, sink,
7104-
window_anchor=False):
7123+
window_anchor=False, initial_capital=None):
71057124
if report.trades or report.trades_len or report.equity_curve or report.equity_curve_len:
71067125
raise _engine_invariant("a report with a sink must contain summary scalars only")
71077126
# Reuse the unchanged producer for its small fields and exact key order.
71087127
# The C ABI supplied no arrays, so this cannot walk native report rows.
71097128
out = build_report_dict(report, ohlcv_path, n_bars, first_ts, last_ts, elapsed,
71107129
applied_inputs, applied_overrides, applied_runtime,
7111-
window_anchor=window_anchor)
7130+
window_anchor=window_anchor,
7131+
initial_capital=initial_capital)
71127132
out["summary"] = _sink_summary(report, sink)
71137133
if getattr(sink, "fd_mode", False):
71147134
_validate_report_result(out) # the engine's summary checked every T
@@ -7120,7 +7140,8 @@ def _build_sink_report(report, ohlcv_path, n_bars, first_ts, last_ts, elapsed,
71207140
] if diagnostics.count_e else ()
71217141
first_bar_trade = ({"entry_bar_index": 0},) if diagnostics.trade_at_start else ()
71227142
out["diagnostics"].update(
7123-
_annualization_block(endpoints, out["metrics"], first_bar_trade, window_anchor))
7143+
_annualization_block(
7144+
endpoints, out["metrics"], first_bar_trade, window_anchor, initial_capital))
71247145
# stdout.json has the small report only. The wrapper/finalizer owns
71257146
# the stream's artifact and receipt; no child-local path names it.
71267147
out.pop("trades")
@@ -7148,7 +7169,8 @@ def _build_sink_report(report, ohlcv_path, n_bars, first_ts, last_ts, elapsed,
71487169
out["equity_curve"] = equity_curve
71497170
first_bar_trade = ({"entry_bar_index": 0},) if diagnostics.trade_at_start else ()
71507171
out["diagnostics"].update(
7151-
_annualization_block(diagnostics, out["metrics"], first_bar_trade, window_anchor))
7172+
_annualization_block(
7173+
diagnostics, out["metrics"], first_bar_trade, window_anchor, initial_capital))
71527174
if sink.named:
71537175
reference = {
71547176
"path": sink.path,
@@ -7334,6 +7356,15 @@ def _sink_outputs_block(lib, state, sink, resources, *, summary_only=False):
73347356
return block
73357357

73367358

7359+
def _read_strategy_initial_capital(lib, state):
7360+
accessor = getattr(lib, "strategy_initial_capital", None)
7361+
if accessor is None:
7362+
return None
7363+
accessor.argtypes = [ctypes.c_void_p]
7364+
accessor.restype = ctypes.c_double
7365+
return accessor(state)
7366+
7367+
73377368
def _discard_stdout() -> None:
73387369
"""Point stdout's file descriptor at /dev/null after a failed report write,
73397370
so what stdout still buffers is dropped instead of written again at exit."""

0 commit comments

Comments
 (0)