Skip to content

Commit 3769703

Browse files
luisleo526claude
andcommitted
Merge origin/main into facts-tokens-20261003
Bring the PR up to date so the strict required-status rule applies to the current base. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2 parents f8f06e1 + f6fdc03 commit 3769703

12 files changed

Lines changed: 505 additions & 36 deletions

‎docs/adr/0001-kernel-adapter-boundary.md‎

Lines changed: 15 additions & 15 deletions
Large diffs are not rendered by default.

‎docs/design/native-feature-parity.md‎

Lines changed: 11 additions & 11 deletions
Large diffs are not rendered by default.

‎docs/pages/pine-to-native.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -534,9 +534,9 @@ range-end reporting is off.
534534
| --- | --- | --- | --- | --- |
535535
| `strategy.account_currency` | `currency` native_run_spec.hpp:639 | `currency` pineforge.h:499 | `hello_kernel.cpp` / `native_fx_roll_strategy.cpp` | Declared, never inferred. `account_fx` native_run_spec.hpp:650 is the one positive scalar that converts quote to account; a timestamped curve is `configure_native_fx_curve` native_host.hpp:1202. The FX-roll example runs a JPY account on a USD-quoted stock through such a curve, and a step of it is a margin check point of its own. |
536536
| `strategy.initial_capital` | `initial_capital` native_run_spec.hpp:648 | `initial_capital` pineforge.h:501 | `hello_kernel.cpp` | The value you declared, unchanged by the run. |
537-
| `strategy.equity` | `native_marked_equity(mark)` native_host.hpp:1301; on the Pine host `current_equity() + open_profit(mark)` pine_strategy_host.hpp:661 | `strategy_native_marked_equity_v1` native_c_api.h:3060 | `native_open_lots_strategy.cpp` | Pine marks at the current `close`. The native marked call takes the mark explicitly and returns balance plus open lots' **net** `unrealized_pnl` native_host.hpp:341, which is Pine's equity: TradingView charges the open lots' paid entry fees. |
538-
| `strategy.netprofit` | on the Pine host `net_profit` pine_strategy_host.hpp:660; `net_profit` engine.hpp:967 is the closed trades' figure | `net_profit` pineforge.h:374, less open lots' `entry_commission` | `native_open_lots_strategy.cpp` | The closed trades' net profit less the entry fees the open lots have paid. The report's figure is the closed trades'. |
539-
| `strategy.netprofit_percent` | derive: `net_profit` pine_strategy_host.hpp:660 over `initial_capital` native_run_spec.hpp:648 | `pf_metrics_t` pineforge.h:320 | `native_sized_report_strategy.cpp` | The kernel keeps the facts, not the ratio. |
537+
| `strategy.equity` | `native_marked_equity(mark)` native_host.hpp:1301; on the Pine host `current_equity() + open_profit(mark)` pine_strategy_host.hpp:664 | `strategy_native_marked_equity_v1` native_c_api.h:3060 | `native_open_lots_strategy.cpp` | Pine marks at the current `close`. The native marked call takes the mark explicitly and returns balance plus open lots' **net** `unrealized_pnl` native_host.hpp:341, which is Pine's equity: TradingView charges the open lots' paid entry fees. |
538+
| `strategy.netprofit` | on the Pine host `net_profit` pine_strategy_host.hpp:663; `net_profit` engine.hpp:967 is the closed trades' figure | `net_profit` pineforge.h:374, less open lots' `entry_commission` | `native_open_lots_strategy.cpp` | The closed trades' net profit less the entry fees the open lots have paid. The report's figure is the closed trades'. |
539+
| `strategy.netprofit_percent` | derive: `net_profit` pine_strategy_host.hpp:663 over `initial_capital` native_run_spec.hpp:648 | `pf_metrics_t` pineforge.h:320 | `native_sized_report_strategy.cpp` | The kernel keeps the facts, not the ratio. |
540540
| `strategy.grossprofit` | `gross_profit` engine.hpp:968 | `pf_trade_stats_t` pineforge.h:269 | `native_open_lots_strategy.cpp` | |
541541
| `strategy.grossprofit_percent` | `grossprofit_percent` engine.hpp:975 | `pf_metrics_t` pineforge.h:320 | `native_sized_report_strategy.cpp` | Of initial capital. |
542542
| `strategy.grossloss` | `gross_loss` engine.hpp:969 | `pf_trade_stats_t::gross_loss` pineforge.h:224 | `native_open_lots_strategy.cpp` | For booked closed trades the protected C++ accessor is signed (negative for losses), while the C report stores a positive loss magnitude. With range-end reporting off, `-C.gross_loss == gross_loss()` and `netprofit == grossprofit - C.gross_loss`. With it on, C trade statistics include the report-only range-end rows; filter `open_at_end == 0` before comparing to Pine's closed-trade statistics. |

‎docs/report-terminal-quote.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Report-only terminal chart quote
2+
3+
Magnified intraday runs can explicitly declare the native chart feed through
4+
`PINEFORGE_RUN_REPORT_CHART_QUOTE` and its `_SHA256` pin. The harness verifies
5+
that it is the original chart input, selects its terminal bar inside the original
6+
inclusive requested range, and passes only its timestamp and close to the Pine
7+
host. It does not register a security feed, change execution bars, or extend the
8+
run. Daily runs keep their existing input contract.
9+
10+
`PineStrategyHost::present_report` projects this quote through the existing F09
11+
hook after completion. Only synthetic `open_at_end` rows change: exit time,
12+
price, net PnL, PnL percent, and commission. Their contribution to displayed net
13+
profit and the final equity point changes correspondingly. The existing report
14+
pipeline then derives display metrics from those projected rows and equity;
15+
these metrics are presentation, not stored execution statistics. Entry fields,
16+
quantity, excursions, bar indexes, actual trades, hashes, trace and diagnostic
17+
counters, and preceding equity points stay unchanged.
18+
19+
The quote is inert during active streams, failed runs, nonmagnified and daily
20+
runs, and when older than the completed mark or more than one chart interval
21+
ahead. Repeated reports are idempotent. This input is deliberately absent from
22+
the broker-state hash: its immutable feed pin belongs to the caller's input
23+
manifest. No runtime C ABI export or codegen change is needed; reserved numeric
24+
metadata keys bridge generated strategies, and direct hosts have an explicit
25+
setter and clearer.
26+
27+
The accepted window has two chart-grid points: the completed mark itself or
28+
exactly one chart interval later. Off-grid timestamps and quotes beyond a
29+
session gap are inert. Direct hosts can inspect
30+
`report_terminal_quote_applied(report)` without mutating state. The harness
31+
returns `report_terminal_quote.applied` from the presented terminal timestamps
32+
and logs `APPLIED` or `INERT` after the run, rather than treating declaration as
33+
application. It reads timestamps only, not the exit price, so a quote stamped at
34+
the completed mark itself, where the terminal timestamps already match without
35+
it, logs `APPLIED` even if the engine declined it. A quote one interval later,
36+
the shape the measured probes use, is reported exactly. A quote persists on a
37+
reused host until explicitly cleared.
38+
39+
Runtime fingerprint provenance includes the declared quote's timestamp, close
40+
and native feed SHA-256, even when inert. ON and OFF fingerprints differ, as do
41+
quotes differing in any of those three values; undeclared fingerprints retain
42+
their old shape. Displayed net profit is summed from the projected rows in
43+
report order, bit-equal to `metrics.all.net_profit`; last equity is initial
44+
capital plus that sum.
45+
46+
## Presentation fields and known limits
47+
48+
Derived report metrics count as presentation only while execution trades,
49+
fills, book and state remain byte-identical.
50+
51+
| Group | Fields allowed to change |
52+
|---|---|
53+
| Synthetic `open_at_end` row | `exit_time`, tick-rounded `exit_price`, `pnl`, `pnl_pct`, `commission` |
54+
| Report / final equity point | `net_profit`; final `time_ms` and `equity` |
55+
| `metrics.all` and the open lot's side | Net/gross profit or loss and percentages, profit factor, average trade/win/loss and percentages, win/loss ratio, largest win/loss and percentages, commission, expectancy; counts, profitable percentage and streaks only on a PnL sign flip |
56+
| `metrics.equity` | Terminal-point drawdown/runup extremes and percentages, monthly/bar Sharpe and Sortino, CAGR, Calmar and recovery factor |
57+
58+
The projected row retains its old `max_runup`, `max_drawdown` and
59+
`exit_bar_index`: the quote does not invent a traversed path or an execution
60+
bar. `buy_hold_return` and its percentage retain the last executed close.
61+
The last equity point is relabelled, not appended, so the ordinary completed
62+
mark disappears and the curve can have a two-interval gap; TradingView has an
63+
additional point. Crossing a month boundary creates a one-point trailing
64+
monthly bucket for Sharpe/Sortino and makes the previous equity point the
65+
closing month's sample. Those derived metrics are not validated against
66+
TradingView by the trade-only grader; only the terminal trade mark is evidenced.

‎include/pineforge/source/pine_strategy_host.hpp‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -408,6 +408,9 @@ class PineStrategyHost : public NativeStrategyHost, public BrokerStateHashProvid
408408
void enable_pine_intraday_cap();
409409
void attach_pine_execution_adapter();
410410
void set_syminfo_metadata(const std::string&, double) override;
411+
bool set_report_terminal_quote(std::int64_t time_ms, double close);
412+
void clear_report_terminal_quote();
413+
bool report_terminal_quote_applied(const ReportC&) const;
411414
// TradingView's session calendar for the chart's symbol (lane XAU-CAL):
412415
// per session day, its open and its close, ascending, each day opening
413416
// before it closes and no later than the next opens. Where it holds a
@@ -948,6 +951,9 @@ class PineStrategyHost : public NativeStrategyHost, public BrokerStateHashProvid
948951
void publish_security_eval_state_at_calling_boundary(SecurityEvalState& state);
949952
void scheduler_record_range_end(const Bar&);
950953
void present_report(ReportC*) const override;
954+
bool report_terminal_quote_eligible() const;
955+
std::int64_t report_quote_time_ms_ = -1;
956+
double report_quote_close_ = std::numeric_limits<double>::quiet_NaN();
951957
// One report point per published source slot. The kernel records it
952958
// (NativeReportPolicy::KernelRecordedAtHostMarks); this host owns only
953959
// the Pine cadence that says where the points fall.

‎scripts/broker_state_hash_waivers.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,5 +66,7 @@ close_margin_follow_up_units_ # Transient within-bar value: the units the follow
6666
symbol_calendar_ # Source-host run input (lane XAU-CAL): TradingView's session calendar for the chart's symbol, installed when the run begins and never changed by it (PineStrategyHost::set_symbol_calendar), as the feed and the syminfo session are; the session flags and time offsets it answers are rewritten before every source callback and fold nowhere.
6767
symbol_calendar_closes_ # C-ABI transport of that calendar (syminfo metadata "symbol_calendar_close:<i>"), read once when the run begins; configuration, never next-decision state.
6868
symbol_calendar_days_ # C-ABI transport of that calendar's day count (syminfo metadata "symbol_calendar_days"), read once when the run begins; configuration, never next-decision state.
69+
report_quote_time_ms_ # Explicit report-only native chart input; read only by present_report after completion, pinned in the caller input manifest, never an execution or next-decision input. test_report_terminal_quote proves equal per-bar hashes and stored trades.
70+
report_quote_close_ # Price of the same report-only native chart input; present_report projects returned rows without changing lots, fills, execution statistics, or hash state. test_report_terminal_quote proves repeated-report and execution identity.
6971
symbol_calendar_opens_ # C-ABI transport of that calendar (syminfo metadata "symbol_calendar_open:<i>"), read once when the run begins; configuration, never next-decision state.
7072
waypoint_chain_bar_ # Transient within-bar value: the script bar whose path took a waypoint margin call (rest_waypoint_margin_call), set on that bar's path and cleared by the same bar's post-script close; compared only for equality with the current script bar, so it is dead at every bar boundary.

‎scripts/fingerprint_self_test.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -789,6 +789,27 @@ def check(name: str, cond: bool) -> None:
789789
check("run_strategy: same effective gate has deterministic digest",
790790
gated_fp["digest"] == same_gate_fp["digest"])
791791

792+
quote_kwargs = dict(runtime_kwargs, report_terminal_quote=(gate_ms, 112.0),
793+
report_terminal_quote_source_sha256="a" * 64)
794+
quote_runtime = rs.build_runtime_provenance(quote_kwargs, gate_ms)
795+
quote_fp = rs.build_fingerprint({"runtime": quote_runtime})["digest"]
796+
check("run_strategy: OFF runtime keeps its existing quote-free shape",
797+
"report_terminal_quote" not in gated_runtime)
798+
check("run_strategy: quote provenance pins time, close and feed digest",
799+
quote_runtime["report_terminal_quote"] == {
800+
"time_ms": gate_ms, "close": 112.0, "source_file_sha256": "a" * 64})
801+
check("run_strategy: quote ON and OFF fingerprints differ",
802+
quote_fp != gated_fp["digest"])
803+
for field, value in (("report_terminal_quote", (gate_ms + 1, 112.0)),
804+
("report_terminal_quote", (gate_ms, 113.0)),
805+
("report_terminal_quote_source_sha256", "b" * 64)):
806+
changed = rs.build_runtime_provenance(dict(quote_kwargs, **{field: value}), gate_ms)
807+
check(f"run_strategy: changing quote {field} changes the fingerprint",
808+
rs.build_fingerprint({"runtime": changed})["digest"] != quote_fp)
809+
check("run_strategy: quote runtime helper is deterministic and pure",
810+
rs.build_runtime_provenance(quote_kwargs, gate_ms) == quote_runtime
811+
and quote_kwargs["report_terminal_quote"] == (gate_ms, 112.0))
812+
792813
# Timestamped FX provenance is pinned to the exact effective C-ABI values,
793814
# not only to a provider-file hash, point count, or endpoints. Changing an
794815
# interior rate must therefore change both the value digest and the final

‎scripts/run_strategy.py‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -690,6 +690,13 @@ def build_runtime_provenance(run_kwargs: dict, trade_start_ms: int | None) -> di
690690
calendar = _session_calendar_provenance(run_kwargs.get("syminfo_metadata"))
691691
if calendar is not None:
692692
runtime["session_calendar"] = calendar
693+
quote = run_kwargs.get("report_terminal_quote")
694+
if quote is not None:
695+
runtime["report_terminal_quote"] = {
696+
"time_ms": int(quote[0]),
697+
"close": float(quote[1]),
698+
"source_file_sha256": run_kwargs.get("report_terminal_quote_source_sha256") or "",
699+
}
693700
return runtime
694701

695702

@@ -2369,6 +2376,8 @@ def run(self, bars_csv: Path, params: dict | None = None,
23692376
aux_security_input_tf: str | None = None,
23702377
aux_security_source_file_sha256: str | None = None,
23712378
native_security_feeds: dict | None = None,
2379+
report_terminal_quote: tuple | None = None,
2380+
report_terminal_quote_source_sha256: str | None = None,
23722381
ohlcv_start_ms: int | None = None,
23732382
ohlcv_end_ms: int | None = None,
23742383
bar_magnifier: bool = False,
@@ -2570,6 +2579,18 @@ def run(self, bars_csv: Path, params: dict | None = None,
25702579
state, str(mkey).encode(), float(mval))
25712580
except (TypeError, ValueError):
25722581
continue
2582+
if report_terminal_quote is not None:
2583+
quote_time, quote_close = report_terminal_quote
2584+
if (isinstance(quote_time, bool) or not isinstance(quote_time, int)
2585+
or not 0 <= quote_time <= 9007199254740991
2586+
or not math.isfinite(quote_close) or quote_close <= 0):
2587+
raise ValueError("report_terminal_quote must be (epoch milliseconds, finite positive close)")
2588+
if not hasattr(self.lib, "strategy_set_syminfo_metadata"):
2589+
raise RuntimeError("strategy library lacks report-only metadata support; rebuild it")
2590+
self.lib.strategy_set_syminfo_metadata(
2591+
state, b"report_terminal_quote_time_ms", float(quote_time))
2592+
self.lib.strategy_set_syminfo_metadata(
2593+
state, b"report_terminal_quote_close", float(quote_close))
25732594
if account_currency_fx_series is not None:
25742595
if not hasattr(self.lib, "strategy_set_account_currency_fx_series"):
25752596
raise RuntimeError(
@@ -2737,12 +2758,29 @@ def run(self, bars_csv: Path, params: dict | None = None,
27372758
result["native_security_feeds"] = native_feed_report
27382759
if probe_requests is not None:
27392760
result["probe_requests"] = probe_requests_provenance(probe_requests)
2761+
if report_terminal_quote is not None:
2762+
result["report_terminal_quote"] = {
2763+
"time_ms": int(report_terminal_quote[0]),
2764+
"close": float(report_terminal_quote[1]),
2765+
"source_file_sha256": report_terminal_quote_source_sha256 or "",
2766+
"applied": _report_terminal_quote_applied(result, report_terminal_quote,
2767+
script_tf or input_tf or ""),
2768+
}
27402769
return result
27412770
finally:
27422771
self.lib.report_free(ctypes.byref(report))
27432772
self.lib.strategy_free(state)
27442773

27452774

2775+
def _report_terminal_quote_applied(report: dict, quote: tuple, script_tf: str) -> bool:
2776+
rows = [row for row in report["trades"] if row.get("open_at_end")]
2777+
times = report["equity_curve_time_ms"]
2778+
return bool(60 < _tf_seconds(script_tf) < 86400
2779+
and report["bar_magnifier_enabled"] and rows and times
2780+
and times[-1] == int(quote[0])
2781+
and all(row["exit_time"] == int(quote[0]) for row in rows))
2782+
2783+
27462784
def _with_tail_bar(bars, n: int, tail: tuple):
27472785
"""``bars[:n]`` followed by the input bar ``tail`` -- ``(timestamp, open,
27482786
high, low, close, volume)`` -- when it is later than the last of them
@@ -3711,6 +3749,8 @@ def _infer_bar_interval_ms(csv_path: Path) -> int:
37113749
# The case runner names the lane's 1m feed (and its sha256) here for every
37123750
# case; a script that declares use_bar_magnifier = true is run on it.
37133751
MAGNIFIER_FEED_ENV = "PINEFORGE_RUN_MAGNIFIER_FEED"
3752+
REPORT_QUOTE_FEED_ENV = "PINEFORGE_RUN_REPORT_CHART_QUOTE"
3753+
REPORT_QUOTE_SHA256_ENV = "PINEFORGE_RUN_REPORT_CHART_QUOTE_SHA256"
37143754
MAGNIFIER_FEED_SHA256_ENV = "PINEFORGE_RUN_MAGNIFIER_FEED_SHA256"
37153755

37163756

@@ -3962,6 +4002,26 @@ def not_run(why: str) -> MagnifierPlan:
39624002
"source_file_sha256": chart_sha}
39634003
detail += f"; the chart's daily bars are its daily feed (sha256 {chart_sha})"
39644004
magnified["native_security_feeds"] = installed
4005+
elif env.get(REPORT_QUOTE_FEED_ENV) or env.get(REPORT_QUOTE_SHA256_ENV):
4006+
quote_path = Path(str(env.get(REPORT_QUOTE_FEED_ENV) or "")).resolve()
4007+
quote_sha = str(env.get(REPORT_QUOTE_SHA256_ENV) or "").strip().lower()
4008+
if not env.get(REPORT_QUOTE_FEED_ENV) or not re.fullmatch(r"[a-f0-9]{64}", quote_sha):
4009+
raise ValueError("report-only native chart quote requires an explicit path and sha256 pin")
4010+
if _sha256_file(quote_path) != quote_sha:
4011+
raise ValueError("report-only native chart quote sha256 does not match its pin")
4012+
if _sha256_file(chart_ohlcv) != quote_sha:
4013+
raise ValueError("report-only native chart quote must be the declared chart feed")
4014+
quote_bars, quote_count, _ = _load_bars(quote_path,
4015+
ohlcv_start_ms=run_kwargs.get("ohlcv_start_ms"),
4016+
ohlcv_end_ms=run_kwargs.get("ohlcv_end_ms"))
4017+
if quote_count == 0:
4018+
raise ValueError("report-only native chart quote has no bar in the inclusive run range")
4019+
terminal = quote_bars[quote_count - 1]
4020+
if not math.isfinite(terminal.close) or terminal.close <= 0:
4021+
raise ValueError("report-only native chart quote close must be finite and positive")
4022+
magnified["report_terminal_quote"] = (int(terminal.timestamp), float(terminal.close))
4023+
magnified["report_terminal_quote_source_sha256"] = quote_sha
4024+
detail += f"; declared report-only native chart quote sha256 {quote_sha} at {_fmt_utc_ms(terminal.timestamp)}"
39654025
return MagnifierPlan("declared", detail, feed, magnified)
39664026

39674027
# --- docker runner (pineforge-release image) ---------------------------
@@ -4396,6 +4456,11 @@ def run_engine(start_ms):
43964456
trade_start_ms = signal_start_ms
43974457
emit_window = (signal_start_ms, emit_window[1])
43984458
report = run_engine(trade_start_ms)
4459+
if run_kwargs.get("report_terminal_quote") is not None:
4460+
quote_result = report.get("report_terminal_quote") or {}
4461+
quote_status = "APPLIED" if quote_result.get("applied") else "INERT"
4462+
print(f" magnifier: report-only native chart quote {quote_status}; "
4463+
f"sha256 {run_kwargs.get('report_terminal_quote_source_sha256', '')}")
43994464
raw_trade_count = len(report["trades"])
44004465
trades_to_write = _filter_trades_to_window(report["trades"], report_window)
44014466
# The trades the verifier grades are the CSV's — run_strategy.py writes

0 commit comments

Comments
 (0)