Skip to content

Commit 783bb39

Browse files
luisleo526claude
andauthored
Apply syminfo.mincontract as the engine's lot grid in the Docker harness (#322)
* Apply syminfo.mincontract as the engine's lot grid in the Docker harness The harness applied only mintick, pointvalue, timezone and session from the --syminfo file, so a percent-of-equity strategy traded sub-lot quantities that the instrument does not allow. A positive finite mincontract is now set as the engine's qty_step metadata (order quantities are floored to it) and as the mincontract metadata, before the other setters. Absent or null changes nothing; any other value, or a library without the metadata setter, fails the run with the structured error line, which shows the value's JSON text. An applied grid is recorded as applied_runtime.syminfo and so in the fingerprint. pineforge-release vendors docker/run_json.py from the engine tag at every release, so the change has to live here to reach a published image: this is the same file as pineforge-release#21, whose copy the next sync overwrites. docker/README.md, the docker/entrypoint.sh header comment and CHANGELOG.md document it; docker/run_json_syminfo_test.py covers it. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * Address review of the lot-grid harness change Tests: restore the checks the port from pineforge-release dropped (SyminfoError is a ValueError; the grid sits in the digested token bytes in canonical form and the plain token has no syminfo) and pin the recorded grid to floats, so an integer mincontract is not recorded as an int. Docs: the error text shows the parsed value re-encoded as JSON, not the file's text; a file that is not JSON or not an object still ends in a traceback; a library without the metadata setter fails the run only when mincontract is set (CHANGELOG); the PINEFORGE_SYMINFO row leaves the "runtime args" table (it is applied through the syminfo setters and has its own section); the extra-keys object is named plainly. docker/run_json.py and docker/entrypoint.sh are unchanged. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> --------- Co-authored-by: Claude Sonnet 5.5 <[email protected]>
1 parent 015dd9a commit 783bb39

5 files changed

Lines changed: 372 additions & 29 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,19 @@ version number follows semantic versioning over the surfaces the
77

88
## Unreleased
99

10+
- **Harness lot grid:** `docker/run_json.py` applies `mincontract` from the
11+
`--syminfo` (`PINEFORGE_SYMINFO`) file as the engine's lot grid, the metadata key
12+
`qty_step` plus the `mincontract` key a script's `syminfo.mincontract` reads, so a
13+
percent-of-equity strategy no longer trades sub-lot quantities. Absent or `null`
14+
changes nothing: the report and its fingerprint are as before, apart from
15+
`elapsed_seconds`. A value that is not a positive finite number, or, when
16+
`mincontract` is set, a strategy library without
17+
`strategy_set_syminfo_metadata`, fails the run (one
18+
`{"engine":"pineforge","error":...}` line, harness exit 1, entrypoint exit 4)
19+
instead of running without the grid. An applied grid is recorded as
20+
`applied_runtime.syminfo`, so a gridded run has its own fingerprint digest. The
21+
engine library is unchanged.
22+
1023
- **Native runner routing and delivery:** webhooks are optional; `--webhook-routes`
1124
adds first-match per-action targets and payload `pineforge-native-order-action/v2`.
1225
New `actions`, `status` and offline `redeliver` commands expose the journal and

‎docker/README.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,41 @@ header) are configured via separate env vars:
137137
| `PINEFORGE_MAGNIFIER_SAMPLES` | `4` | Sub-bar sample count when magnifier is on (≥2) |
138138
| `PINEFORGE_MAGNIFIER_DIST` | `endpoints` | `uniform`, `cosine`, `triangle`, `endpoints`, `front_loaded`, `back_loaded` |
139139

140+
### Instrument metadata (`PINEFORGE_SYMINFO`)
141+
142+
`PINEFORGE_SYMINFO` (the harness's `--syminfo`) names a JSON file holding the
143+
instrument's metadata, either a flat object or `{"syminfo": {...}}`; an object
144+
with extra keys (for example a full instrument record) can be passed as is, keys
145+
the harness does not use are ignored. It applies `mincontract` first, then
146+
`mintick`, `pointvalue`, `timezone` and `session`, each through the strategy
147+
library's `strategy_set_syminfo_*` setters.
148+
149+
- `mincontract` is the instrument's lot size (TradingView's
150+
`syminfo.mincontract`). It is set as the engine's `qty_step` metadata, so order
151+
quantities are floored to that grid, and as `mincontract` metadata, so a
152+
script's `syminfo.mincontract` reads return the same value.
153+
- Absent or `null`: no lot grid, and the run is the same as without the key.
154+
- Any other value that is not a positive finite JSON number (`0`, `-1`,
155+
`"0.001"`, `true`, `NaN`, `Infinity`, a list or an object) fails the run
156+
before it starts: one line `{"engine":"pineforge","error":"syminfo.mincontract
157+
must be a positive finite number, got <value>"}` on stdout, where `<value>` is
158+
the parsed value re-encoded as JSON (so `1e-400` shows as `0.0`) cut to 80
159+
characters (`got true`, `got "0.001"`), harness exit status 1, entrypoint exit
160+
4. A strategy library without `strategy_set_syminfo_metadata` fails the same
161+
way when `mincontract` is set: the harness never runs without the grid it was
162+
given. A file that is not valid JSON, or not a JSON object, is not covered by
163+
this: the harness ends in a Python traceback, as before.
164+
- An applied grid is recorded as `applied_runtime.syminfo`
165+
(`{"qty_step": <v>, "mincontract": <v>}`) and so in
166+
`fingerprint.provenance.runtime`: its fingerprint digest differs from the
167+
gridless run's. Without a grid the key is absent and the report, fingerprint
168+
included, is what it was before this key was supported, apart from
169+
`elapsed_seconds` (and, with `--bench`, the timing samples). `mintick`,
170+
`pointvalue`, `timezone` and `session` are not recorded.
171+
172+
`docker/run_json.py` is vendored: pineforge-release copies it from the engine tag
173+
at every release, so the lot-grid handling (`mincontract`) lives in this file.
174+
140175
The engine catches every error (TF mismatch, unsupported emulation
141176
flags, unknown-input-TF, etc.) into `strategy_get_last_error()`; the
142177
container surfaces these as `{"engine":"pineforge","error":"..."}` on

‎docker/entrypoint.sh‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,14 @@
3636
# PINEFORGE_MAGNIFIER_DIST Sample distribution: uniform / cosine / triangle /
3737
# endpoints (default) / front_loaded / back_loaded.
3838
#
39+
# Optional env var (instrument metadata):
40+
# PINEFORGE_SYMINFO Path to a syminfo JSON file, a flat object or {"syminfo": {...}}.
41+
# Keys applied: mincontract (the lot size: order quantities are
42+
# floored to it), then mintick, pointvalue, timezone, session; other
43+
# keys are ignored. mincontract absent or null: no lot grid. Any other
44+
# mincontract that is not a positive finite number fails the run
45+
# (exit 4, one {"engine":"pineforge","error":...} line on stdout).
46+
#
3947
# Exit codes:
4048
# 0 success (JSON report, or C++ in transpile-only mode, on stdout)
4149
# 2 missing input mount
@@ -131,7 +139,7 @@ echo "[pineforge] running backtest ..." >&2
131139
# PINEFORGE_TRADE_START_MS unix-ms; suppress orders before it
132140
# PINEFORGE_CHART_TZ IANA tz for date builtins
133141
# PINEFORGE_MAGNIFIER_VOLUME_WEIGHTED 1/true → vw magnifier (needs BAR_MAGNIFIER)
134-
# PINEFORGE_SYMINFO path to a syminfo.json
142+
# PINEFORGE_SYMINFO path to a syminfo.json (see the header)
135143
# PINEFORGE_BENCH (+_WARMUP/_REPEATS) 1/true → timing mode
136144
extra=()
137145
[[ -n "${PINEFORGE_TRADE_START_MS:-}" ]] && extra+=(--trade-start-ms "${PINEFORGE_TRADE_START_MS}")

‎docker/run_json.py‎

Lines changed: 94 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,13 @@
7474
}
7575
}
7676
77+
applied_runtime (and so provenance.runtime) also holds
78+
"syminfo": {"qty_step": float, "mincontract": float} when --syminfo set a lot
79+
grid from syminfo.mincontract; without one the key is absent.
80+
81+
A run error or a --syminfo the harness rejects (see apply_syminfo) prints one
82+
line {"engine": "pineforge", "error": "<text>"} instead, exit status 1.
83+
7784
NaN convention: any metric with an empty/zero denominator is null (JSON has no
7885
NaN); a real computed 0 stays 0. See the report-schema + metrics reference docs
7986
for the per-field meaning of every metrics.* key.
@@ -834,6 +841,10 @@ def load_strategy(so_path: Path) -> ctypes.CDLL:
834841
for _n in ("strategy_set_syminfo_timezone", "strategy_set_syminfo_session"):
835842
if hasattr(lib, _n):
836843
getattr(lib, _n).argtypes = [ctypes.c_void_p, ctypes.c_char_p]
844+
if hasattr(lib, "strategy_set_syminfo_metadata"):
845+
lib.strategy_set_syminfo_metadata.argtypes = [
846+
ctypes.c_void_p, ctypes.c_char_p, ctypes.c_double]
847+
lib.strategy_set_syminfo_metadata.restype = None
837848

838849
# Validation-parity setters mirrored from scripts/run_strategy.py. All
839850
# hasattr-guarded: trade_start_time + chart_timezone are runtime PF exports;
@@ -854,16 +865,52 @@ def load_strategy(so_path: Path) -> ctypes.CDLL:
854865
return lib
855866

856867

868+
class SyminfoError(ValueError):
869+
"""A --syminfo file the harness cannot apply as given. main() reports it as
870+
the structured {"engine", "error"} failure (exit 1), never as a traceback."""
871+
872+
873+
# This file is vendored: pineforge-release copies it from the pineforge-engine
874+
# tag at every release. Keep the lot-grid handling (mincontract) identical in
875+
# both repos, or a sync drops it.
857876
def apply_syminfo(lib, strat, syminfo_path):
858877
"""Apply syminfo.json (data-worker schema) via strategy_set_syminfo_*.
859-
Tolerant: missing keys skipped. Accepts {"syminfo": {...}} or a flat dict."""
878+
Tolerant: missing keys skipped. Accepts {"syminfo": {...}} or a flat dict.
879+
880+
mincontract (TradingView's syminfo.mincontract, the instrument's lot size)
881+
is strict: absent or null applies nothing; anything else must be a positive
882+
finite JSON number, else SyminfoError before any setter runs. A valid one is
883+
set first, as the metadata key qty_step (the engine floors order quantities
884+
to that grid) and as mincontract (what syminfo.mincontract reads return).
885+
Returns what main() records in applied_runtime["syminfo"]:
886+
{"qty_step": v, "mincontract": v}, or {} when no grid was applied."""
860887
import json
861888
doc = json.loads(open(syminfo_path).read())
862889
si = doc.get("syminfo", doc)
890+
applied = {}
891+
lot = si.get("mincontract")
892+
if lot is not None:
893+
try:
894+
v = (float(lot) if isinstance(lot, (int, float))
895+
and not isinstance(lot, bool) else math.nan)
896+
except OverflowError: # an int beyond binary64
897+
v = math.nan
898+
if not (math.isfinite(v) and v > 0):
899+
raise SyminfoError(
900+
"syminfo.mincontract must be a positive finite number, got "
901+
+ json.dumps(lot)[:80])
902+
if not hasattr(lib, "strategy_set_syminfo_metadata"):
903+
raise SyminfoError(
904+
"the strategy library has no strategy_set_syminfo_metadata, "
905+
"so syminfo.mincontract cannot be applied")
906+
lib.strategy_set_syminfo_metadata(strat, b"qty_step", v)
907+
lib.strategy_set_syminfo_metadata(strat, b"mincontract", v)
908+
applied = {"qty_step": v, "mincontract": v}
863909
if "mintick" in si: lib.strategy_set_syminfo_mintick(strat, float(si["mintick"]))
864910
if "pointvalue" in si: lib.strategy_set_syminfo_pointvalue(strat, float(si["pointvalue"]))
865911
if si.get("timezone"): lib.strategy_set_syminfo_timezone(strat, str(si["timezone"]).encode())
866912
if si.get("session"): lib.strategy_set_syminfo_session(strat, str(si["session"]).encode())
913+
return applied
867914

868915

869916
def fmt_utc(ms: int) -> str:
@@ -1149,17 +1196,28 @@ def main() -> int:
11491196
# Volume-weighted magnifier only meaningful when the magnifier is on.
11501197
vw_on = bool(args.magnifier_volume_weighted) and bar_magnifier == 1
11511198

1199+
# The lot grid the last _make_state() applied (the body run's), for
1200+
# applied_runtime["syminfo"]; {} when none.
1201+
syminfo_applied: dict = {}
1202+
11521203
def _make_state():
11531204
"""Create + fully configure a fresh strategy state — everything EXCEPT the
11541205
timed run_backtest_full call. Mirrors scripts/run_strategy.py's setup so the
11551206
engine behaves identically to the ctypes validation harness."""
1207+
nonlocal syminfo_applied
11561208
st = lib.strategy_create(b"{}")
11571209
for k, v in inputs.items():
11581210
lib.strategy_set_input(st, k.encode(), v.encode())
11591211
for k, v in overrides.items():
11601212
lib.strategy_set_override(st, k.encode(), v.encode())
11611213
if args.syminfo:
1162-
apply_syminfo(lib, st, args.syminfo)
1214+
try:
1215+
# A replacement apply_syminfo may return None (or another non-dict).
1216+
r = apply_syminfo(lib, st, args.syminfo)
1217+
syminfo_applied = r if isinstance(r, dict) else {}
1218+
except SyminfoError:
1219+
lib.strategy_free(st)
1220+
raise
11631221
if args.trade_start_ms is not None and hasattr(lib, "strategy_set_trade_start_time"):
11641222
lib.strategy_set_trade_start_time(st, int(args.trade_start_ms))
11651223
if args.chart_tz and hasattr(lib, "strategy_set_chart_timezone"):
@@ -1181,31 +1239,38 @@ def _run(st, rep):
11811239
# isolates the engine hot loop (closest to the GBench harness). dlopen
11821240
# already happened above (load_strategy), outside any loop.
11831241
timing = None
1184-
if args.bench:
1185-
warmup = max(0, int(args.warmup))
1186-
repeats = max(1, int(args.repeats))
1187-
for _ in range(warmup):
1188-
st = _make_state(); rep = ReportC()
1189-
try:
1190-
_run(st, rep)
1191-
finally:
1192-
lib.report_free(ctypes.byref(rep)); lib.strategy_free(st)
1193-
samples_ns: list[int] = []
1194-
for _ in range(repeats):
1195-
st = _make_state(); rep = ReportC()
1196-
try:
1197-
t0 = time.perf_counter_ns(); _run(st, rep); t1 = time.perf_counter_ns()
1198-
samples_ns.append(t1 - t0)
1199-
finally:
1200-
lib.report_free(ctypes.byref(rep)); lib.strategy_free(st)
1201-
timing = _timing_block(
1202-
samples_ns, warmup=warmup, repeats=repeats,
1203-
bar_magnifier=bar_magnifier, magnifier_samples=magnifier_samples,
1204-
magnifier_dist=args.magnifier_dist.strip().lower() or "endpoints",
1205-
volume_weighted=vw_on)
1206-
1207-
# --- Body run: one configured run for trades / metrics / diagnostics. ---
1208-
state = _make_state()
1242+
try:
1243+
if args.bench:
1244+
warmup = max(0, int(args.warmup))
1245+
repeats = max(1, int(args.repeats))
1246+
for _ in range(warmup):
1247+
st = _make_state(); rep = ReportC()
1248+
try:
1249+
_run(st, rep)
1250+
finally:
1251+
lib.report_free(ctypes.byref(rep)); lib.strategy_free(st)
1252+
samples_ns: list[int] = []
1253+
for _ in range(repeats):
1254+
st = _make_state(); rep = ReportC()
1255+
try:
1256+
t0 = time.perf_counter_ns(); _run(st, rep); t1 = time.perf_counter_ns()
1257+
samples_ns.append(t1 - t0)
1258+
finally:
1259+
lib.report_free(ctypes.byref(rep)); lib.strategy_free(st)
1260+
timing = _timing_block(
1261+
samples_ns, warmup=warmup, repeats=repeats,
1262+
bar_magnifier=bar_magnifier, magnifier_samples=magnifier_samples,
1263+
magnifier_dist=args.magnifier_dist.strip().lower() or "endpoints",
1264+
volume_weighted=vw_on)
1265+
1266+
# --- Body run: one configured run for trades / metrics / diagnostics. ---
1267+
state = _make_state()
1268+
except SyminfoError as e:
1269+
# A rejected --syminfo: the structured failure, before any stdout.
1270+
json.dump({"engine": "pineforge", "error": str(e)},
1271+
sys.stdout, separators=(",", ":"))
1272+
sys.stdout.write("\n")
1273+
return 1
12091274
report = ReportC()
12101275
started = time.time()
12111276
try:
@@ -1234,6 +1299,8 @@ def _run(st, rep):
12341299
"trade_start_ms": args.trade_start_ms,
12351300
"chart_tz": args.chart_tz or "",
12361301
}
1302+
if syminfo_applied:
1303+
applied_runtime["syminfo"] = syminfo_applied
12371304
incarnation_accessor = getattr(
12381305
lib, "strategy_closed_trade_entry_incarnation", None)
12391306
trade_entry_incarnations = (

0 commit comments

Comments
 (0)