Repository navigation
Backtest time grows quadratically with the number of distinct entry ids (~200× on 700k bars) #452
Description
Summary
A strategy that gives every new position its own strategy.entry id (for example an id built from bar_index or the bar time) runs about 200 times slower than the same strategy with a fixed id per side, with identical trades. Run time grows at least with the square of the number of distinct ids, so long intraday histories become impractical for parameter sweeps.
Versions
- pineforge-engine
693fd9e6(VERSION 1.6.1), pineforge-codegen 1.6.1 - macOS 15.8, Apple M1 Pro, Apple clang 17.0.0, strategy built with
-O2, engine Release (-O3)
Reproduction
Two scripts that differ only in their order ids:
//@version=6
strategy('Unique entry ids', overlay=true, initial_capital=10000, default_qty_type=strategy.fixed, default_qty_value=0.01)
fast = ta.ema(close, 20)
slow = ta.ema(close, 60)
string n = str.tostring(bar_index)
if ta.crossover(fast, slow)
strategy.close_all()
strategy.entry('Long ' + n, strategy.long)
strategy.exit('Long stop ' + n, 'Long ' + n, stop = close * 0.97)
if ta.crossunder(fast, slow)
strategy.close_all()
strategy.entry('Short ' + n, strategy.short)
strategy.exit('Short stop ' + n, 'Short ' + n, stop = close * 1.03)
The control uses the fixed ids 'Long', 'Long stop', 'Short' and 'Short stop'.
Feed: Binance spot BTCUSDT 5-minute bars, one run_backtest_full per measurement, report time excluded.
| Span | Bars | Trades | Unique ids | Fixed ids |
|---|---|---|---|---|
| 2020 | 104,862 | 1,966 | 3.38 s | — |
| 2020–2021 | 209,767 | 3,979 | 14.49 s | — |
| 2020-01-01 → 2026-09-07 | 702,505 | 13,500 | 268.3 s | 1.22 s |
Both variants produce the same 13,500 trades and the same net profit.
Where the time goes
macOS sample for 6 s during the 2020–2021 unique-id run (4,743 samples, all inside run_backtest_full):
- about 60% in
PineExecutionAdapter::source_entry_slot_count(), called fromPineStrategyHost::scheduler_publish_source_baron every bar; - about 24% in
PineExecutionAdapter::erase_retired_rows, called fromon_bar_open.
Nearly all of that time is std::unordered_map<std::string, CohortFacts>::find plus the string hash and memcmp it calls. This is consistent with a walk over every entry id seen so far on each bar.
Expected
Per-bar cost independent of how many distinct ids the run has used so far.
Workaround we use
Keep one id per side for strategy.entry / strategy.exit and carry any per-trade label elsewhere (for example in the alert message or the comment). On the strategies we checked (no pyramiding), TradingView's Deep Backtesting report lists the same trades before and after the change.
Activity
[-]Backtest time grows quadratically with the number of distinct entry ids (~100× on 700k bars)[/-][+]Backtest time grows quadratically with the number of distinct entry ids (~200× on 700k bars)[/+]luisleo526 commentedon Oct 25, 2026
Thank you for the clear report and the profile. We re-ran your two scripts on the public Binance spot BTCUSDT 5-minute feed: your "Unique entry ids" script, and the control built from it with the four fixed ids from your description ('Long', 'Long stop', 'Short', 'Short stop'). Engine time of
run_backtest_fullonly; seconds rounded to 2 decimals; unique = mean of 3 runs, fixed = mean of 4.Notes on the setup:
What the numbers show:
We are leaving this issue open until the per-bar cost is flat.
engine 1.8.0: cohort tables with per-event work independent of pendin…
luisleo526 commentedon Oct 26, 2026
Engine 1.8.0 (27cd77b, #541) removes two scaling terms for runs with many pending orders: scratch seeks on a full close go from N(N+1)/2 to 2N+1 and per-event cohort walks visit only cohorts that hold an opening or live units. It does not remove the per-bar Theta(R) retire pass, and the dense shape still does not finish within 600 s from 10,000 bars (numbers in the comment on #458). Remaining here: the retire pass and a profile of the dense run past 4,000 bars.
letiep97 commentedon Oct 30, 2026
Thank you for the measurements and for 1.8.0. With the unique-id script the dense case is now close to the fixed-id one, which was the main problem for us.
The fixed-id control is still slower than on 1.6.x in 1.9.0, on both platforms we have. Same two scripts from this issue, same Binance BTCUSDT 5-minute feed 2020-01-01 to 2026-09-07 (702,505 bars), time of
run_backtest_fullonly, median of 3 runs, Release builds from the tags:On fixed ids 1.9.0 takes 11.8 times as long as 1.6.1 on macOS and 8.6 times on Linux. Your 10-25 run had 3.6 times for 1.7.0 against 1.6.3 on x86-64; we have no x86-64 machine to check 1.9.0 there.
Trades (13,505) and net profit are identical between the two engines and between the two platforms.
On macOS,
sampleputs 75% of the 1.9.0 fixed-id run inPineExecutionAdapter::on_applied, almost all of it underExitScratchScope/ExitScratchRows::putonCommandScratchFile:preadandpwritetake 44% of the run andmemmove31%, behind the 64 KiB cache (kCacheBytesinsrc/command_scratch_cache.hpp). The extra time follows the number of fills, about 0.3 ms per fill on macOS and 0.4 ms on Linux (13,505 trades, so about 27,000 fills), not the number of ids. Settingstrategy_set_scratch_tables_v1to bounded memory did not change the time.Two questions:
luisleo526 commentedon Oct 31, 2026
Thanks for the pull request in #458: the parts of its approach we took landed in 1.8.0 (#541).
We re-measured your two scripts (distinct entry ids, and the same script with fixed ids as the control) on the published 1.9.0 image and on a build made from the 1.9.1 source. Data: BTCUSDT 5-minute bars (2020-01-01 to 2026-09-07), the whole 702,834-bar span and its first 10,000 and 100,000 bars; Linux, each run pinned to one core; three runs per cell, both builds on the same machine. The times are the wall-clock time of a run through the hosted runner up to its outcome (median of the three runs, with the fastest and slowest in brackets). At 100,000 and 702,834 bars that runner refuses the report as too large (it exceeds 8 MiB), so those times are the time until the run ended with that refusal. These runs were not stopped at the hosted runner's 30-second limit: at 702,834 bars all three runs of the distinct-id script on 1.9.0 took longer than 30 s (33.11 to 63.80 s); none of the 1.9.1 runs did (26.93 to 27.39 s).
What this shows:
Limit of the current fix: A cohort's target read uses a fixed 64 KiB index (4,096 open lots). With more simultaneously open lots, with intrabar or observed-tick input, or inside the source host's applied callback, it uses the original, slower reader.
We will report here again when the remaining growth is addressed.
luisleo526 commentedon Nov 4, 2026
Thanks for the report and the two scripts. This is fixed on the current engine main (the 1.10.0 line).
We re-ran your two scripts (unique
strategy.entryids vs the same script with the ids fixed) on Binance BTCUSDT 5m, on one machine, building v1.6.3 and current main side by side. The table shows the engine's own elapsed time, the minimum of 3 runs; the trade lists were identical across every run of the same span.On v1.6.3 the cost of unique ids grows quadratically with the number of bars (12x, 28x, 181x). On current main the unique-id run stays about 1.2-1.3x the fixed-id run, does not grow with size, and the full span with unique ids runs 47x faster than on v1.6.3 (523.5 s -> 11.15 s).
The fixed-id run itself is about 3x slower on current main than on v1.6.3 (for example 2.89 s -> 8.71 s on the full span). That is a separate regression in the base cost per bar, and we are tracking it.
Closing as fixed. If you still see the slowdown on 1.10.0 or later, please reopen with the script and the bar count.