You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit a02160f
Browse filesBrowse the repository at this point in the historyBrowse files
Bench: parity re-run and PineForge re-timed at engine 063e446 -- all 201 trade lists unchanged, 89 ms / 603k bars/s, 15x vs PyneCore (lane BENCH3)
Parity: every dylib and the GBench harness rebuilt on main 063e446; all 201
pineforge_trades.csv are byte-identical to the e9ad37d refresh, so summary.md,
trade_comparison.md and indicator_comparison.md regenerate unchanged (corpus
100/100 excellent, closed 100 excellent + 1 strong, slot 181 unchanged).
Speed: PineForge re-timed in two GBench batches (slots 001-100 at load 5.41,
101-201 at 5.90): median 89.45 ms, 603k bars/s (Q1 425k, Q3 731k), median
1.02x the e9ad37d time per strategy. PyneCore, PineTS and vectorbt are not
re-measured; their timing files reproduce the e9ad37d report byte for byte.
The throughput package is not re-timed: the host was never quiet.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Copy file name to clipboardExpand all lines: README.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -33,7 +33,7 @@ The separate PineForge compiler, [`pineforge-codegen`](https://github.com/pinefo
33
33
34
34
-**Proven, not promised.** All 4,190 probes — 312 open reference strategies plus 413 real community scripts on 15 markets and timeframes — grade *excellent* or *strong* against TradingView's own trade lists: **4,182 excellent, 8 strong, zero moderate**. The current full sweep evaluates 2,819,967 TradingView trades, with 2,818,237 matched by the verifier.
35
35
-**Open runtime.** The engine and native live runner are Apache-2.0. The separately distributed [PineForge compiler](https://github.com/pineforge-4pass/pineforge-codegen-oss/blob/main/LICENSE) uses PolyForm Noncommercial terms with additional personal-trading permission; commercial use requires a separate license. Public reference strategies, benchmarks and validation tooling are available in their respective repositories; the community-script test set is not redistributed.
36
-
-**Fast.** In-process, no interpreter: median **15× faster than PyneCore** on 196 timed strategies (a median 611k bars/s per strategy with the bar magnifier on). Parameter sweeps re-run a loaded `.so` with new inputs — no recompile, no fork.
36
+
-**Fast.** In-process, no interpreter: median **15× faster than PyneCore** on 196 timed strategies (a median 603k bars/s per strategy with the bar magnifier on). Parameter sweeps re-run a loaded `.so` with new inputs — no recompile, no fork.
37
37
-**Deterministic to the bit.** Two runs with the same inputs produce identical trade lists. Same on Linux and macOS.
38
38
-**Yours to embed.** 97 `extern "C"` functions across two headers — 65 to run a compiled strategy, 32 to drive the kernel yourself — append-only ABI. Call it from C, Python, Rust, Go, Node, Julia — or let an AI agent drive it over MCP.
39
39
@@ -311,20 +311,20 @@ Four slots have no PyneCore trade list:
- **3 runtime errors:** PyneCore raises a `RuntimeError` in its `request.security` engine on slots 140, 166 and 197. It hits the same error intermittently on a fourth slot, 143, which could not be timed.
313
313
314
-
**Speed** was measured on a quiet host ([`benchmarks/results/speed.md`](benchmarks/results/speed.md)):
314
+
**Speed** was measured on a quiet host ([`benchmarks/results/speed.md`](benchmarks/results/speed.md)). PineForge was re-timed at engine `063e4460`. PyneCore, vectorbt and PineTS were not re-measured; their figures come from the same host's earlier 2026-09-22 window, at engine `e9ad37dd`.
315
315
316
-
- **PineForge:** a median of 88 ms per strategy over the feed, in-process with the bar magnifier on (611k bars/s; quartiles 434k–726k).
317
-
- **PyneCore:** a median of 1,475 ms per subprocess (36.6k bars/s). The median per-strategy speedup is **15×** across the 196 strategies both engines time (p5 6×, p95 70×).
316
+
- **PineForge:** a median of 89 ms per strategy over the feed, in-process with the bar magnifier on (603k bars/s; quartiles 425k–731k).
317
+
-**PyneCore:** a median of 1,475 ms per subprocess (36.6k bars/s). The median per-strategy speedup is **15×** across the 196 strategies both engines time (p5 6×, p95 68×).
318
318
-**vectorbt:** a median of 102 ms for its 13 ports; PineForge is 1.3× faster on the same 13.
319
319
-**PineTS:** 486 ms for the canonical 10-indicator script.
320
-
- **Throughput package** ([`benchmarks/throughput/`](benchmarks/throughput/)): the magnifier-off hot loop runs at a median of **0.61 M bars/s** per strategy (N=201, median of five quiet runs).
320
+
-**Throughput package** ([`benchmarks/throughput/`](benchmarks/throughput/)): the magnifier-off hot loop runs at a median of **0.61 M bars/s** per strategy (N=201, median of five quiet runs). Those runs were measured at `e9ad37dd`; they were not re-timed at `063e4460` because the host was never quiet.
321
321
322
322
**Not comparable with the 2026-06-11 table** (PineForge 100/100, PyneCore 85/100, 162×):
323
323
324
324
-**Tiers:** they now come from the canonical `scripts/verify_corpus.py::analyze_strategy` rubric. The old table graded a different 100-strategy population with `compare.py`'s own copy of the rubric, which had drifted from the canonical one and no longer parsed the current tape format.
325
-
- **Speed:** the ratio fell because the engine is slower per bar, not because the host changed. The 2026-06-11 engine, rebuilt on the same host, reproduces its June timings, whilethe current engine is 12–20× slower on the probes both populations share ([provenance](benchmarks/results/speed.md#provenance)).
325
+
-**Speed:** the ratio fell because the engine is slower per bar, not because the host changed. The 2026-06-11 engine, rebuilt on the same host, reproduced its June timings, while engine `e9ad37dd` was 12–20× slower on the probes both populations share. That A/B was measured at `e9ad37dd`; `063e4460` times those probes within 7 % of it ([provenance](benchmarks/results/speed.md#provenance)).
326
326
327
-
Last refresh **2026-09-22** (engine `e9ad37dd`, PyneCore 6.10.2, PineTS 0.9.34, vectorbt 0.28.2, Apple M4 Max). Per-strategy table: [`benchmarks/results/summary.md`](benchmarks/results/summary.md). Population manifest: [`benchmarks/results/selection.md`](benchmarks/results/selection.md). Method, fairness and the reproduction recipe: [`benchmarks/README.md`](benchmarks/README.md).
327
+
Last refresh **2026-09-22** (engine `063e4460`, PyneCore 6.10.2, PineTS 0.9.34, vectorbt 0.28.2, Apple M4 Max). Per-strategy table: [`benchmarks/results/summary.md`](benchmarks/results/summary.md). Population manifest: [`benchmarks/results/selection.md`](benchmarks/results/selection.md). Method, fairness and the reproduction recipe: [`benchmarks/README.md`](benchmarks/README.md).
Copy file name to clipboardExpand all lines: benchmarks/README.md
+8-6Lines changed: 8 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ This directory compares **PineForge** with two open-source PineScript runtimes a
8
8
9
9
## Headline
10
10
11
-
Last refresh **2026-09-22**. Versions: engine `main``e9ad37dd` running the committed `generated.cpp` (codegen `89645d6`), PyneCore 6.10.2, PineTS 0.9.34 and vectorbt 0.28.2, on an Apple M4 Max.
11
+
Last refresh **2026-09-22**. Versions: engine `main``063e4460` running the committed `generated.cpp` (codegen `89645d6`), PyneCore 6.10.2, PineTS 0.9.34 and vectorbt 0.28.2, on an Apple M4 Max.
12
12
13
13
**Population: 200 strategies in 201 slots**, selected by [`select_population.py`](select_population.py) with seed 20260921. The manifest is [`results/selection.md`](results/selection.md).
14
14
@@ -30,22 +30,23 @@ Each engine's trade list is graded against TradingView's own export (266,451 tra
30
30
31
31
Counting the 200 strategies (slot `201` in place of `192`), PineForge grades 199 excellent and 1 strong. PyneCore grades 133 excellent, 42 strong, 8 moderate, 12 weak and 2 minimal, and 3 slots have no trade list.
32
32
33
-
Speed was measured on a quiet host, re-checked before every timing batch; the details are in [`results/speed.md`](results/speed.md):
33
+
Speed was measured on a quiet host, re-checked before every timing batch; the details are in [`results/speed.md`](results/speed.md). PineForge was re-timed at engine `063e4460`. PyneCore, vectorbt and PineTS were not re-measured: their rows come from the same host's earlier 2026-09-22 window, at engine `e9ad37dd`.
34
34
35
35
| Engine | How it is timed | Strategies | Median per strategy | Bars/s: Q1 · **median** · Q3 |
36
36
|---|---|---:|---:|---|
37
-
| PineForge | in-process Google Benchmark, bar magnifier on | 201 |88.3 ms |434k · **611k** · 726k|
37
+
| PineForge | in-process Google Benchmark, bar magnifier on | 201 |89.4 ms |425k · **603k** · 731k|
| vectorbt | in-process, the 13 ports that load | 13 | 102.3 ms | — |
40
40
| PineTS | subprocess wall time of the canonical 10-indicator script | 1 | 485.8 ms | — |
41
41
42
-
- PineForge's median speedup is **15× over PyneCore**, per strategy across the 196 strategies both engines time (p5 6×, p95 70×), and 1.3× over vectorbt across its 13 ports.
43
-
- The throughput package ([`throughput/`](throughput/)) measures the magnifier-off hot loop at a median of **0.61 M bars/s** per strategy. That figure is over all 201 slots, and is the median of five quiet runs.
42
+
- PineForge's median speedup is **15× over PyneCore**, per strategy across the 196 strategies both engines time (p5 6×, p95 68×), and 1.3× over vectorbt across its 13 ports.
43
+
- The throughput package ([`throughput/`](throughput/)) measures the magnifier-off hot loop at a median of **0.61 M bars/s** per strategy. That figure is over all 201 slots, and is the median of five quiet runs at engine `e9ad37dd`. It was not re-timed at `063e4460` because the host was never quiet.
44
+
- Per strategy, the `063e4460` sweep takes 1.02× the `e9ad37dd` sweep's time at the median (p5 0.95×, p95 1.07×), at a higher host load (5.41–5.90 against 3.79).
44
45
45
46
**These numbers are not comparable with the 2026-06-11 table** (PineForge 100/100 excellent, PyneCore 85/100, 162×):
46
47
47
48
-**Tiers:** that table graded a different 100-strategy population with `compare.py`'s own copy of the rubric. The copy had drifted from the canonical rubric and no longer parsed the current tape format. `compare.py` now calls the canonical rubric directly, and PyneCore moved from 6.4.6 to 6.10.2 in between.
48
-
-**Speed:** the ratio fell because the engine is slower per bar, not because the host changed. The 2026-06-11 engine, rebuilt on this host in the same window, reproduces its June timings. On the three probes both populations share, the current engine is 12–20× slower with the magnifier on; see[the provenance](results/speed.md#provenance).
49
+
-**Speed:** the ratio fell because the engine is slower per bar, not because the host changed. The 2026-06-11 engine, rebuilt on this host in the `e9ad37dd` window, reproduced its June timings. On the three probes both populations share, engine `e9ad37dd` was 12–20× slower with the magnifier on. That A/B was measured at `e9ad37dd` and not re-run; `063e4460` times the same probes within 7 % of `e9ad37dd`. See[the provenance](results/speed.md#provenance).
0 commit comments