Skip to content

Commit a4ab775

Browse files
luisleo526claude
andcommitted
docs: source-verified runtime-overrides + order-execution cheatsheet
Reference for driving the engine and building a forward/real-time executor: Part 1 catalogs every runtime-overrideable parameter (C-ABI setters, strategy() overrides, risk params, TF/magnifier, SymInfo, harness inputs.json); Part 2 documents the order-execution rules (per-bar sequence, fill-price table, intrabar path, slippage/commission, entry state machine + carry, pyramiding, sizing, exits/OCA, risk gates, trade stamping) + a 12-point forward-replay gotchas checklist. Every claim adversarially verified against source (file:line). Notes the syminfo setters / get_input_source land with PR #25/#18. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
1 parent cd08969 commit a4ab775

1 file changed

Lines changed: 306 additions & 0 deletions

File tree

Lines changed: 306 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,306 @@
1+
# PineForge Engine — Runtime Overrides & Order-Execution Cheatsheet
2+
3+
> **Source-verified.** Every claim was adversarially checked against the code
4+
> (file:line) on `feat/input-source-syminfo-issues` (engine) /
5+
> `feat/input-source-color-syminfo` (codegen). Corrections from that pass are
6+
> folded in (notably: `engine_chart_timezone` is validator-only not a harness
7+
> key; the `pending_close_qty_in_bar_` reset is in the outer bar loop not
8+
> `dispatch_bar`; stop-entry mintick rounding is skipped on gap fills;
9+
> `PERCENT_OF_EQUITY` includes open profit only on the explicit-`qty_type` path).
10+
11+
Reference for driving the engine and for building a **forward / real-time**
12+
executor that matches backtest semantics. Two parts:
13+
14+
1. [Runtime-overrideable parameters](#part-1--runtime-overrideable-parameters)
15+
2. [Order-execution rules](#part-2--order-execution-rules) (the hard part)
16+
17+
> Surface note: `get_input_source`, `strategy_set_syminfo_timezone/_session/_metadata`
18+
> land with the input.source/syminfo feature (engine PR #25 + codegen PR #18).
19+
> Everything else is on `main`. File:line refs are approximate — grep the symbol.
20+
21+
---
22+
23+
# Part 1 — Runtime-overrideable parameters
24+
25+
"Runtime" = settable on a strategy handle **before `run_backtest*`**, without
26+
recompiling. "Compile-time" = baked into `generated.cpp` from the Pine
27+
`strategy(...)` / `strategy.risk.*` source; only changeable by re-transpiling.
28+
29+
## 1.1 C-ABI setters (call before run; persist across runs on the same handle)
30+
31+
| Setter | Default | Semantics |
32+
|---|---|---|
33+
| `strategy_set_input(s, key, value)` | — | Override a Pine `input.*()` by **title** (key = input title string). Value serialized as string. |
34+
| `strategy_set_override(s, key, value)` | — | Override a `strategy()` decl param (see 1.2 for the 9 keys). |
35+
| `strategy_set_chart_timezone(s, tz)` | `""`=UTC | Chart **display** TZ (IANA). Drives `hour/minute/dayofweek/...` builtins **and intraday-cap day rollover**. `""`/`UTC`/`Etc/UTC` = fast `gmtime_r`; else process-global mutex. |
36+
| `strategy_set_trade_start_time(s, ms)` | `INT64_MIN` (no gate) | Suppress `entry/order/exit/close` before this Unix-ms; TA still runs. |
37+
| `strategy_set_trace_enabled(s, on)` | `0` | Capture `// @pf-trace` records into the report. |
38+
| `strategy_set_magnifier_volume_weighted(s, on)` | `0` | Volume-weighted sub-bar sampling (only if magnifier on). |
39+
| `strategy_set_syminfo_timezone(s, tz)` | `UTC` | Exchange TZ → feeds `session.ismarket` / `time(session)`. **Separate slot** from chart tz. |
40+
| `strategy_set_syminfo_session(s, str)` | `24x7` | Session string e.g. `0930-1600:23456` (days 1=Sun..7=Sat). |
41+
| `strategy_set_syminfo_metadata(s, key, double)` | reads `na` | Inject a fundamental field (`shares_outstanding_total`, `target_price_*`, …); engine only computes — data fed externally. |
42+
43+
Run entry points: `run_backtest(s,bars,n,out)` (auto-detect TF, no magnifier) /
44+
`run_backtest_full(s,bars,n,input_tf,script_tf,bar_magnifier,magnifier_samples,magnifier_dist,out)`.
45+
Always check `strategy_get_last_error(s)` after. `report_free(&out)` to free.
46+
47+
**Persistence:** `inputs_`, `chart_timezone_`, `trade_start_time_`,
48+
`trace_enabled_`, overrides, syminfo all persist across runs on the same handle.
49+
**Risk state is NOT auto-reset** between runs on a reused handle.
50+
51+
## 1.2 `strategy()` params — runtime-overrideable via `strategy_set_override`
52+
53+
Baked from the `.pine` at compile-time; these 9 can be overridden at runtime.
54+
Sentinels for "not overridden": `NaN` (doubles), `-1` (ints).
55+
56+
| Key | Default | Values |
57+
|---|---|---|
58+
| `initial_capital` | 1,000,000 | decimal |
59+
| `commission_value` | 0 | decimal |
60+
| `commission_type` | percent(0) | `percent`/`0`, `cash_per_order`/`1`, `cash_per_contract`/`2` |
61+
| `default_qty_value` | 1 | decimal |
62+
| `default_qty_type` | fixed(0) | `fixed`/`0`, `percent_of_equity`/`1`, `cash`/`2` |
63+
| `pyramiding` | 1 | int |
64+
| `slippage` | 0 | int (ticks) |
65+
| `process_orders_on_close` | false | `true`/`1`/`false`/`0` |
66+
| `close_entries_rule` | FIFO | `ANY`/`any`/`1` = ANY, else FIFO |
67+
68+
**Compile-time only (no override path):** `margin_long` / `margin_short`
69+
(default 100 = 1x), `script_has_strategy_close_` (gates the deferred-flip rule).
70+
71+
## 1.3 `strategy.risk.*` — compile-time only
72+
73+
Set inside `on_bar` by generated code; no ABI setter. All default to
74+
unlimited/both:
75+
`max_intraday_filled_orders`, `max_intraday_loss` (+is_pct), `max_drawdown`
76+
(+is_pct), `max_position_size`, `allow_entry_in` (direction lock),
77+
`max_cons_loss_days`. Gate: `check_risk_allow_entry`; halt latch:
78+
`update_risk_state`.
79+
80+
## 1.4 Timeframe / magnifier (via `run_backtest_full`)
81+
82+
| Param | Default | Notes |
83+
|---|---|---|
84+
| `input_tf` | `""` auto-detect | feed TF (`1`,`5`,`15`,`60`,`1D`…) |
85+
| `script_tf` | = input_tf | chart TF; must be ≥ input_tf (aggregates up) |
86+
| `bar_magnifier` | off | enable intrabar sub-bar walk |
87+
| `magnifier_samples` | 4 | ticks/parent bar (synthetic mode) |
88+
| `magnifier_dist` | ENDPOINTS(3) | 0 UNIFORM,1 COSINE,2 TRIANGLE,3 ENDPOINTS,4 FRONT_LOADED,5 BACK_LOADED. With **real** sub-bars, forced to ENDPOINTS + 4 samples. |
89+
90+
## 1.5 SymInfo fields
91+
92+
`ticker, tickerid, currency, basecurrency, type, timezone(UTC), session(24x7),
93+
volumetype(base), description, mintick(0.01), pointvalue(1.0)`. `mintick` drives
94+
slippage + directional rounding. Set via the `SymInfo`-taking `run()` overload,
95+
or `timezone`/`session`/metadata via the C-ABI setters in 1.1.
96+
97+
## 1.6 Harness (`scripts/run_strategy.py`) — `inputs.json` keys
98+
99+
Any non-meta key → `strategy_set_input`. Recognized: `ohlcv_csv`,
100+
`ohlcv_start_ms`, `script_tf`, `input_tf`, `chart_timezone`, and a
101+
`runtime_overrides` block: `bar_magnifier`, `magnifier_distribution`,
102+
`magnifier_samples`, `magnifier_volume_weighted`, `timezone`, `session`,
103+
`syminfo_metadata`. `tv_trades_csv` / `tv_trades_csv_tz` are verify-only.
104+
105+
> `engine_chart_timezone` is **NOT** a `run_strategy.py` key — it's consumed
106+
> only by the private canonical validator. In this harness use `chart_timezone`.
107+
> (Verified: `engine_chart_timezone` appears nowhere in `scripts/run_strategy.py`.)
108+
109+
---
110+
111+
# Part 2 — Order-execution rules
112+
113+
For a forward executor: the engine is a per-bar state machine. Below is the
114+
exact sequence, fill math, and the subtleties that bite on real-time replay.
115+
116+
## 2.1 Per-bar order of operations
117+
118+
`dispatch_bar()`:
119+
120+
**`process_orders_on_close = false` (default):**
121+
> Step 1 runs in the **outer bar loop** (`engine_run.cpp:96 / 367 / 443`),
122+
> *before* `dispatch_bar()` — `dispatch_bar()` itself does not reset it.
123+
```
124+
1. pending_close_qty_in_bar_ = 0 # reset (outer loop, pre-dispatch_bar)
125+
2. process_pending_orders(bar) # OLD stop/limit/market from prior bars
126+
a. update_risk_state() # may latch risk_halted_
127+
b. update_trail_best_for_bar_open(bar)
128+
c. sort_exit_siblings_by_path_fill(bar)
129+
d. sort_orders_by_fill_phase(bar) # gap(0) before intrabar(1)
130+
e. dual-pass opposing-stop loop (pass 0 / pass 1)
131+
f. purge exit orders if now FLAT
132+
3. update_per_trade_extremes() # MFE/MAE current before strategy reads
133+
4. on_bar(bar) # strategy logic; creates NEW orders
134+
5. update_equity_extremes()
135+
# NEW market orders wait for NEXT bar's open
136+
```
137+
138+
**`process_orders_on_close = true`:** identical 1–4, then a **step 4b
139+
`process_pending_orders(bar)`** so NEW market orders fill at **this bar's
140+
close**. New *priced* (stop/limit/trail) orders created this bar are always
141+
skipped from that second pass (they wait for next bar).
142+
143+
**Magnifier:** per sub-bar, per sample tick → `process_pending_orders` +
144+
`update_per_trade_extremes`; `on_bar` runs **once**, on the last tick of the
145+
last sub-bar (`is_first_tick_` forced true there).
146+
147+
## 2.2 Fill price by order type
148+
149+
All comparisons **non-strict** (`<=`/`>=`). After slippage, fills snap to
150+
mintick directionally (buys ceil, sells floor).
151+
152+
| Order | Trigger | Fill price | Gap-at-open |
153+
|---|---|---|---|
154+
| Market | always | `close` (POOC on) / `open` (POOC off) | — |
155+
| Long limit entry | `low ≤ L` | `min(open, L)` | open if `open ≤ L` |
156+
| Short limit entry | `high ≥ L` | `max(open, L)` | open if `open ≥ L` |
157+
| Long stop entry | `high ≥ S` | `max(open, S)`, then ceil-mintick **if fill>open** (skipped on gap) | open if `open ≥ S` |
158+
| Short stop entry | `low ≤ S` | `min(open, S)`, then floor-mintick **if fill<open** (skipped on gap) | open if `open ≤ S` |
159+
| Stop-limit entry | path crosses S then L | activation pt or L | complex (§2.4) |
160+
| Long stop exit | `low ≤ S` | `S` (or open if gap) | open if `open ≤ S` |
161+
| Long limit exit | `high ≥ L` | `L` (or open if gap) | open if `open ≥ L` |
162+
| Short stop exit | `high ≥ S` | `S` (or open if gap) | open if `open ≥ S` |
163+
| Short limit exit | `low ≤ L` | `L` (or open if gap) | open if `open ≤ L` |
164+
| Trail exit (long) | path descends through `best − offset` | trail level (or open if gap) | open if `open ≤ trail` |
165+
166+
**Gap shortcut suppressed on the entry bar** unless the magnifier is active
167+
(then every sub-bar open is a fresh gap event).
168+
169+
## 2.3 Market-order timing (critical)
170+
171+
- POOC **off**: market from `on_bar` → `pending_orders_`, fills **next bar
172+
open**.
173+
- POOC **on**: market from `on_bar` (no stop/limit) → `execute_market_entry`
174+
**immediately at this bar's close** (never queued). `strategy.close` likewise
175+
closes immediately at close.
176+
- Priced order from `on_bar`: queued, evaluated from **next bar** at step 2.
177+
178+
## 2.4 Intrabar price path
179+
180+
- Direction: `|open−high| < |open−low|` ⇒ path `O→H→L→C`, else `O→L→H→C`
181+
(proximity, not candle color).
182+
- 4 waypoints; same-segment priority **STOP > TRAIL > LIMIT**; within a segment,
183+
parametric position `t` decides; tie → STOP first.
184+
- Long position: stop active on **falling** segments, limit on **rising** (short
185+
reversed).
186+
- Two opposing entry stops from flat: `dual_entry_stop_path_winner` picks who
187+
fires first; loser deferred to pass 1; tie → long first. On pass 1, a trailing
188+
short-stop fills as a bracket close; a trailing long-stop after `ShortFirst` is
189+
discarded (TV only treats a same-bar second touch as a bracket exit when the
190+
buy-stop leads).
191+
192+
## 2.5 Slippage & commission
193+
194+
- Slippage = `slippage_ × mintick`; buy +slip then ceil-mintick, sell −slip then
195+
floor-mintick. `slippage_=0` still applies directional mintick rounding.
196+
- Commission per fill: `percent` = `price·qty·v/100`; `cash_per_order` = `v`;
197+
`cash_per_contract` = `v·qty`. **Both entry and exit** commissions deducted in
198+
`emit_close_trade`.
199+
- Raw PnL: long `(exit−entry)·qty`, short `(entry−exit)·qty`.
200+
201+
## 2.6 Entry / position state machine
202+
203+
`execute_market_entry` dispatch:
204+
- FLAT → `enter_market_from_flat`
205+
- same side → `add_to_pyramid_market`
206+
- flat-issued entry while opposite open (`close_only_opposite`) →
207+
`close_opposite_then_enter` (TV: flat-issued bracket entry **closes**, not
208+
reverses)
209+
- opposite → `flip_market_position_to` (new_size = explicit qty)
210+
211+
**Deferred-flip carry (`tv_carry_qty`, `|old|+qty`):** a **priced** entry firing
212+
**from flat** opposite to its `created_position_side` opens at `carry + base_qty`
213+
**iff** all hold: `script_has_strategy_close_`, priced entry, `carry > 0`,
214+
opposite direction. Carry snapshot taken **at `strategy.entry` call time** =
215+
`max(0, position_qty_ − pending_close_qty_in_bar_)`. Same-id replacement
216+
re-snaps carry; siblings in the same cycle get carry zeroed on fire.
217+
218+
**`strategy.entry` vs `strategy.order`:** entry respects pyramiding + margin
219+
gate + deferred-flip; order is `RAW_ORDER`, no margin check, no carry
220+
subtraction.
221+
222+
## 2.7 Pyramiding
223+
224+
Fill blocked if `position_entry_count_ ≥ pyramiding_` **except**:
225+
`flat_armed_priced` (placed from flat, now same dir) and
226+
`pre_armed_opposite_priced` (placed while opposite). One priced ENTRY per bar
227+
(`priced_entry_filled_this_bar_`) unless flat-armed/opposite-sibling.
228+
229+
## 2.8 Sizing
230+
231+
`FIXED`=value; `PERCENT_OF_EQUITY`=`equity·v/100 / fill`; `CASH`=`v / fill`.
232+
**Two equity definitions** (verified): explicit `qty_type=percent_of_equity` via
233+
`strategy.entry` → `calc_qty_for_type`, `equity = initial_capital_ +
234+
net_profit_sum_ + open_profit(close)` (**includes open profit**); the default
235+
`calc_qty` path → `equity = initial_capital_ + net_profit_sum_` (closed PnL
236+
only). `max_position_size` blocks at fill when `position_qty_ ≥ cap`.
237+
238+
## 2.9 Exits, brackets, OCA
239+
240+
- `strategy.exit` → `OrderType::EXIT` bound to `from_entry`; same-id replaces
241+
(keeps queue slot); partial exits one-shot per live position per id.
242+
- `strategy.close[_all]`: immediate at close if POOC/immediately, else deferred
243+
`__close__<id>` filling next open; bumps `pending_close_qty_in_bar_`; a full
244+
close cancels prior-bar adds to the closing position (keeps same-bar reversal
245+
entries).
246+
- FIFO (default) drains `pyramid_entries_` front-to-back; ANY closes only
247+
matching `entry_id`.
248+
- Same-bar multi-level: exit siblings path-sorted (earliest touch; full before
249+
partial on trail; tie = decl order) **before** the phase sort.
250+
- OCA cancel(1): full-fill of a sibling erases the group. OCA reduce(2): sibling
251+
qty reduced by filled; `strategy.exit` with oca → type 1.
252+
253+
## 2.10 Same-bar close-then-entry
254+
255+
`pending_close_qty_in_bar_` accumulates `strategy.close*` qty during `on_bar`;
256+
a later `strategy.entry` sees post-close carry. Full-close market exit sorts
257+
**before** opposite-direction priced entry so the engine is flat when the
258+
deferred-flip entry fires. Reset to 0 at bar start.
259+
260+
## 2.11 Risk gates — when & what
261+
262+
- `update_risk_state()` at **start of `process_pending_orders`** (not
263+
placement): drawdown halt, intraday-loss halt, cons-loss-day halt. Once
264+
`risk_halted_`, blocks all entries.
265+
- `check_risk_allow_entry` at fill (and in `execute_market_entry`): halt,
266+
direction lock, max_position_size.
267+
- Margin gate at **placement** (market entry w/ explicit qty only):
268+
`|qty|·signal_close·margin%/100 ≤ equity(closed)` else silently dropped.
269+
- `max_intraday_filled_orders`: counted at fill, keyed `dayofmonth*100+month` in
270+
**chart tz**; Nth fill emits a synthetic "Close Position (max filled orders)"
271+
exit then **latches** until day rollover; while latched, fills **and
272+
placements** dropped. Cap-close price = `bar.high/low` if the trigger was an
273+
intrabar stop, else `fill_price`.
274+
275+
## 2.12 Trade record & stamping
276+
277+
`emit_close_trade` (single close convergence): `entry_time = pe.time`,
278+
`exit_time = current_bar_.timestamp`, slippage-adjusted prices, PnL net of both
279+
commissions, `entry_bar_index`/`exit_bar_index`, MFE/MAE from the pyramid entry.
280+
`net_profit_sum_`, win/loss/even counts, intraday PnL, cons-loss-day all updated
281+
here. exit_id/comment stamped by the caller post-fill.
282+
283+
## 2.13 Time/index stamping — magnifier caveat
284+
285+
`bar_index_` = script-bar index (fixed across the whole magnifier loop).
286+
`current_bar_.timestamp` under the magnifier = the **sub-bar** timestamp → a
287+
fill mid-sub-bar stamps `entry_time`/`exit_time` with that sub-bar's time, not
288+
the script-bar open. (Aggregation-path label uses first-present sub-bar
289+
timestamp — see `engine_run.cpp` dispatch comment.)
290+
291+
---
292+
293+
## Forward / real-time replay — gotchas checklist
294+
295+
1. Reset `pending_close_qty_in_bar_ = 0` **before** running strategy logic each bar.
296+
2. Call `update_risk_state()` before the fill loop, not at placement.
297+
3. Keep the **two** sequential sorts (exit-sibling path sort, then phase sort).
298+
4. Snapshot `tv_carry_qty` at command-issue time, not fill time.
299+
5. `trail_best_price_` updates at bar open, again after each entry fill.
300+
6. On a flip, un-slip raw price then re-slip per new direction (exit uses old dir, entry new dir).
301+
7. Gap-at-open shortcut is OFF on the entry bar unless magnifier active.
302+
8. OCA-cancel only after a sibling **fully** fills (partial TP keeps SL alive).
303+
9. Intraday-cap latch resets only on **chart-tz** day rollover.
304+
10. Under LTF/real-time, stamp fills with the **sub-bar** time, not the aggregate bar.
305+
11. Market vs priced timing: market next-open (POOC off) / this-close (POOC on); priced waits a bar.
306+
12. Reused handle keeps risk state — reset/recreate per session if you don't want carryover.

0 commit comments

Comments
 (0)