Skip to content

Commit e59c09f

Browse files
luisleo526claude
andcommitted
Merge origin/main (#325, #327) into rel/1.1.0-notes
Brings the branch up to date with main 4f2a475 (#325 7b59662, #327 4f2a475) so the release notes merge last, as the merge order now asks. Conflicts resolved neutrally; the next commit does the editing: - CHANGELOG.md: this branch's 1.1.0 section, with main's two new Unreleased bullets ("Confirmed-bar streams compute what the batch computes:" from #325, "Harness symbol feeds:" from #327) kept verbatim inside it; main's "## Unreleased" heading and its older copies of the bullets this branch already rewrote are not taken. - README.md: #325's known-issue lines replace this branch's "Known issue (v1.0.0, v1.0.1, v1.1.0)" line from a80485c. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2 parents 83d64dc + 4f2a475 commit e59c09f

69 files changed

Lines changed: 10857 additions & 309 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,19 @@ accessor and a replace-options word) are not in this release.
115115
of another key refuses *excellent*. Trade matching and the thresholds are
116116
unchanged; the README's *Distinct-entry identity* section states the rule.
117117

118+
- **Harness symbol feeds:** `docker/run_json.py --symbol-feeds` (`PINEFORGE_SYMBOL_FEEDS`)
119+
installs other symbols' bars for `request.security` on another symbol: a JSON
120+
index keyed by the exact symbol string the script passes and by timeframe, each
121+
feed an OHLCV CSV plus the symbol's catalog `syminfo`, through
122+
`strategy_set_symbol_facts` / `strategy_set_symbol_feed` (engine and codegen
123+
1.0.0 and later). One feed per requested timeframe: nothing aggregates another
124+
symbol's bars, so a `1` feed serves neither a `240` nor a `D` request. Unset
125+
changes nothing; an index or feed it cannot install fails
126+
the run (one `{"engine":"pineforge","error":...}` line, harness exit 1,
127+
entrypoint exit 4). What was installed is recorded as
128+
`applied_runtime.symbol_feeds`, so such a run has its own fingerprint digest.
129+
The engine library is unchanged.
130+
118131
- **Harness lot grid ([#322]).** `docker/run_json.py` applies `mincontract`
119132
from the `--syminfo` (`PINEFORGE_SYMINFO`) file as the engine's lot grid, the
120133
metadata key `qty_step` plus the `mincontract` key a script's
@@ -134,6 +147,17 @@ accessor and a replace-options word) are not in this release.
134147
hub copies `docker/run_json.py` from the engine tag, so `pineforge-release`
135148
1.1.0 carries it.
136149

150+
- **Confirmed-bar streams compute what the batch computes:** a native stream fed
151+
confirmed bars now applies the same TradingView-parity rules as a batch backtest over
152+
the same bars. Rules that were switched off in any stream are now chosen by the input's
153+
domain: historical semantics for batch and confirmed-bar input, observed-print semantics
154+
only for genuine tick input. Fixes the v1.0.0/v1.0.1 known issue where same-chart
155+
`request.security` stopped updating after the first realtime bar when the input and
156+
script timeframes were equal, and two stream-only fill differences (both-sided stop
157+
entries with default sizing; `process_orders_on_close` attribution). Batch results are
158+
unchanged. The streaming guide gains a "Backtest vs live" section listing what a
159+
backtest can see that a live run cannot.
160+
137161
- **Native runner routing and delivery ([#320]).** Webhooks are optional;
138162
`--webhook-routes` adds first-match per-action targets and payload
139163
`pineforge-native-order-action/v2`.

‎README.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -222,9 +222,11 @@ the [native runner guide](runner/README.md) for feed modes, symbol metadata,
222222
feed format, recovery and execution limitations. The existing validation
223223
scoreboard below describes batch backtests; it does not certify new native
224224
live behavior or real broker fills.
225-
Known issue (v1.0.0, v1.0.1, v1.1.0): a stream whose input and script timeframes are
226-
equal serves stale `request.security` values after the first realtime bar; see
227-
[Streaming known issues](docs/pages/streaming.md) for the workaround.
225+
Known issue (v1.0.0, v1.0.1): a stream whose input and script timeframes are
226+
equal serves stale `request.security` values after the first realtime bar.
227+
Fixed on main (#325); included in the next release. See
228+
[Streaming known issues](docs/pages/streaming.md#streaming_known_issues) for the
229+
workaround for those versions.
228230

229231
## Validation scoreboard
230232

‎docker/README.md‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,88 @@ library's `strategy_set_syminfo_*` setters.
172172
`docker/run_json.py` is vendored: pineforge-release copies it from the engine tag
173173
at every release, so the lot-grid handling (`mincontract`) lives in this file.
174174

175+
### Other symbols' bars (`PINEFORGE_SYMBOL_FEEDS`)
176+
177+
A script that calls `request.security` on another symbol reads that symbol's
178+
own bars, never the chart's. Without them, or when the index below lacks the
179+
requested symbol string or timeframe, the run stops where the request's value
180+
is read (`request.security(...) at line N: no data is pinned for this request,
181+
and its value was read`, exit 4). `PINEFORGE_SYMBOL_FEEDS` (the
182+
harness's `--symbol-feeds`) names a JSON index of those bars, installed through
183+
the library's `strategy_set_symbol_facts` and `strategy_set_symbol_feed` (engine
184+
1.0.0 and later):
185+
186+
```json
187+
{"symbols": {
188+
"BINANCE:ETHUSDT": {
189+
"syminfo": {"tickerid": "BINANCE:ETHUSDT", "type": "crypto", "currency": "USDT",
190+
"mintick": 0.01, "session": "24x7", "timezone": "UTC"},
191+
"feeds": {"240": "ethusdt-240.csv", "1D": "ethusdt-1D.csv"}}}}
192+
```
193+
194+
```bash
195+
docker run --rm \
196+
-v $(pwd)/strategy.pine:/in/strategy.pine:ro \
197+
-v $(pwd)/btcusdt-240.csv:/in/ohlcv.csv:ro \
198+
-v $(pwd)/symbols:/in/symbols:ro \
199+
-e PINEFORGE_SYMBOL_FEEDS=/in/symbols/symbols.json \
200+
pineforge > report.json
201+
```
202+
203+
- A symbol key is the exact string the script passes at run time, exchange
204+
prefix and suffix included: `BINANCE:ETHUSDT`, `ETHUSDT` and
205+
`BINANCE:ETHUSDT.P` are three symbols. For `input.symbol` it is the input's
206+
value (its default, or the `PINEFORGE_INPUTS` override). A string naming the
207+
chart's own market is another symbol too: the harness does not set the chart's
208+
`syminfo.tickerid`.
209+
- One feed per timeframe the script requests, keyed in the engine's spelling:
210+
whole minutes as a bare integer (`"240"`, never `"4h"`), else `<n>D|W|M|S`;
211+
a bare `D`/`W`/`M`/`S` is folded to `1D`/`1W`/`1M`/`1S`. A request at
212+
`timeframe.period` (or `""`) reads the feed at the chart's timeframe.
213+
- A feed is a CSV like `ohlcv.csv` (`timestamp,open,high,low,close,volume`; an
214+
empty volume is a symbol that publishes none), paths relative to the index.
215+
Each bar's close is its open plus the timeframe (calendar months for `M`),
216+
right for a 24x7 symbol; give a session-bound symbol a `time_close` column
217+
(unix ms; an empty cell falls back to open plus timeframe). Bars before the
218+
chart's first bar are delivered as history on it; bars after its last are
219+
never read. A header-only feed installs the symbol without bars: its requests
220+
read na on every bar (a symbol with no bars in the window).
221+
- `syminfo` is the symbol's catalog object, flat or `{"syminfo": {...}}`. Its
222+
`type`, `timezone`, `session`, `currency` and `mintick` are set as the
223+
symbol's facts, which `syminfo.*` reads inside the request; `tickerid` is set
224+
as its `canonical` fact, which no `syminfo.*` reads; other keys are ignored.
225+
Inside the request `syminfo.tickerid` is always the key and `syminfo.ticker`
226+
the key after its last `:`. Without `syminfo`, `syminfo.mintick` reads NaN
227+
and those four strings read empty.
228+
- Merge rule (TradingView's): with `lookahead` off a chart bar reads the latest
229+
requested bar whose close is at or before the chart bar's close; with it on,
230+
the latest that opened at or before the chart bar's open. A missing requested
231+
bar carries the last value forward (`gaps` off) or reads na (`gaps` on).
232+
- Limits: at most 256 symbols and 256 feeds in one index. The chart must be its
233+
own input (`PINEFORGE_SCRIPT_TF` unset or equal to the input timeframe), else
234+
the run fails with `request.security of another symbol needs the chart's own
235+
bars as input; input '<i>' aggregated to chart '<s>' is not supported`. The run
236+
is historical only. Each feed is a full pass of its request's expression over
237+
its bars. `request.security_lower_tf` on another symbol reads no feed.
238+
- A feed must be at the timeframe it serves. Nothing aggregates another
239+
symbol's bars: the engine looks a feed up by the exact symbol string and
240+
timeframe, and `strategy_set_symbol_feed` installs the bars as given. A `240`
241+
feed does not serve a `D` request, and a `1` feed serves only a request at
242+
`1`, so give each requested timeframe its own bars.
243+
- An index or feed the harness cannot install (for example: not JSON, a bad
244+
timeframe spelling, two feeds at one timeframe, a CSV without `close`,
245+
timestamps that do not strictly increase, a close after the next bar's open,
246+
a non-positive `mintick`, more than 256 feeds, a library without the setters,
247+
a feed the engine refuses)
248+
fails the run before it starts: one line
249+
`{"engine":"pineforge","error":"--symbol-feeds: ..."}` on stdout, harness exit
250+
1, entrypoint exit 4.
251+
- What was installed is recorded as `applied_runtime.symbol_feeds` (each
252+
symbol's facts, and per feed its bar count, first and last open and a hash of
253+
its values), so the fingerprint digest differs from a run without it. Unset,
254+
or an index naming no symbol: the key is absent and the report is what it was
255+
before this variable existed, apart from `elapsed_seconds`.
256+
175257
The engine catches every error (TF mismatch, unsupported emulation
176258
flags, unknown-input-TF, etc.) into `strategy_get_last_error()`; the
177259
container surfaces these as `{"engine":"pineforge","error":"..."}` on

‎docker/entrypoint.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,14 @@
4343
# keys are ignored. mincontract absent or null: no lot grid. Any other
4444
# mincontract that is not a positive finite number fails the run
4545
# (exit 4, one {"engine":"pineforge","error":...} line on stdout).
46+
# PINEFORGE_SYMBOL_FEEDS Path to a JSON index of other symbols' bars for
47+
# request.security on another symbol: {"symbols": {"<symbol string>":
48+
# {"syminfo": {...}, "feeds": {"<timeframe>": "<csv path>"}}}}, the
49+
# symbol string exactly as the script passes it, one CSV per
50+
# timeframe the script requests (paths relative to the index). Unset:
51+
# nothing is installed and such a request stops the run where its
52+
# value is read. An index or feed the harness cannot install fails
53+
# the run (exit 4, one {"engine":"pineforge","error":...} line).
4654
#
4755
# Exit codes:
4856
# 0 success (JSON report, or C++ in transpile-only mode, on stdout)
@@ -140,12 +148,14 @@ echo "[pineforge] running backtest ..." >&2
140148
# PINEFORGE_CHART_TZ IANA tz for date builtins
141149
# PINEFORGE_MAGNIFIER_VOLUME_WEIGHTED 1/true → vw magnifier (needs BAR_MAGNIFIER)
142150
# PINEFORGE_SYMINFO path to a syminfo.json (see the header)
151+
# PINEFORGE_SYMBOL_FEEDS path to other symbols' feed index (see the header)
143152
# PINEFORGE_BENCH (+_WARMUP/_REPEATS) 1/true → timing mode
144153
extra=()
145154
[[ -n "${PINEFORGE_TRADE_START_MS:-}" ]] && extra+=(--trade-start-ms "${PINEFORGE_TRADE_START_MS}")
146155
[[ -n "${PINEFORGE_CHART_TZ:-}" ]] && extra+=(--chart-tz "${PINEFORGE_CHART_TZ}")
147156
[[ "${PINEFORGE_MAGNIFIER_VOLUME_WEIGHTED:-}" =~ ^(1|true|yes|on)$ ]] && extra+=(--magnifier-volume-weighted)
148157
[[ -n "${PINEFORGE_SYMINFO:-}" ]] && extra+=(--syminfo "${PINEFORGE_SYMINFO}")
158+
[[ -n "${PINEFORGE_SYMBOL_FEEDS:-}" ]] && extra+=(--symbol-feeds "${PINEFORGE_SYMBOL_FEEDS}")
149159
if [[ "${PINEFORGE_BENCH:-}" =~ ^(1|true|yes|on)$ ]]; then
150160
extra+=(--bench --warmup "${PINEFORGE_WARMUP:-3}" --repeats "${PINEFORGE_REPEATS:-20}")
151161
fi

0 commit comments

Comments
 (0)