Benchmarks: 200-strategy refresh (100 corpus + 100 closed), every engine re-timed, README numbers - #274

… time (lane BENCH1, step 2)

The benchmark population now lives in two roots: the public assets
submodule (the 100 corpus slots) and a maintainer-local closed root,
benchmarks/assets-closed/strategies (100 TradingView-scraped scripts that
are never published; gitignored since step 1). Every consumer discovers
both, additively, and a public checkout without the closed root runs the
public slots alone:

- paths.py: CLOSED_STRATEGIES + STRATEGY_ROOTS;
- CMakeLists.txt: the bench section adds the closed root's CMakeLists when
  present (its targets join bench_strategies); the bench options stay OFF
  in every CI profile;
- speed/pineforge_bench.cpp + speed/CMakeLists.txt: GBench enumerates
  BENCH_STRATEGIES_DIR then BENCH_CLOSED_STRATEGIES_DIR;
- run_all.sh: slot loops over both roots (the PyneCore glob no longer drops
  3-digit slots), JOBS-parallel parity runs, per-slot run_strategy.args,
  per-slot _<engine>_error.log, an optional vectorbt step, the PineForge
  canonical indicator run gated by SKIP_PINEFORGE, and quiet_gate: the
  1-min load (optionally waited below QUIET_LOAD_MAX, no cmake --build /
  ctest / ci_verify) recorded before every timing batch;
- compare.py grades every engine with the canonical rubric itself,
  scripts/verify_corpus.py::analyze_strategy (the old copy of the rubric
  was stale and could not parse the current tape schema), reports n/a
  reasons, tallies per root, and --engines/--pending/--out-dir/--*-name
  stage one engine's grades without touching results/summary.md;
- compare_indicators.py: --out and --pending-pineforge;
- speed/time_pynecore.py: both roots, --slots chunks, --out;
- speed/aggregate.py: PineForge optional (pending), the GBench
  with_magnifier entries selected explicitly, host loads, dynamic
  header (date, engine, PyneCore/PineTS versions, feed bars), bars/s for
  every engine (Q1/median/Q3), speedup p5/p95, --out.

No engine source changes.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…hly security contexts; runs chunk by slot (lane BENCH1, step 2)

- runners/run_pynecore.py maps D/W/M (and the 1D/1W/1M spellings) to the
  chart feed with pyne run --security: PyneCore 6.10 serves intraday HTFs
  from the chart data but needs an explicit base file for D/W/M, which it
  resamples -- the bars PineForge aggregates its own HTF series from.
  Without it every script with a daily request.security failed with "No
  OHLCV data found for security context"; an unused mapping leaves the
  output byte-identical (checked on a non-MTF slot).
- run_all.sh: SLOTS narrows the parity loops to slot-number ranges (runs
  can be chunked), SKIP_INDICATORS skips the three canonical indicator
  runs, and quiet_gate counts build/test processes by executable name
  (cmake --build, ctest, python ci_verify.py) -- a full-command grep also
  matched agent processes whose arguments merely quote those commands.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

… stays 0.28.2 (lane BENCH1, step 3)

- benchmarks/pyproject.toml + uv.lock: pynesys-pynecore[cli] >= 6.10.2
  (locked 6.10.2, the current release; the runner's --security mapping
  needs it). The venv is built on CPython 3.12.12, the interpreter of the
  2026-06-11 sweep.
- benchmarks/package.json: pinets ^0.9.34 (the current release; the
  previous refresh ran 0.9.16 under ^0.9.13). The canonical indicator run
  reproduces the committed canonical_pinets.csv byte-for-byte.
- vectorbt stays pinned at 0.28.2: it still installs and runs on the 3.12
  venv (1.1.0 is the current release and was not needed).

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

… run reports and finishes (lane BENCH1, step 2)

slot_dirs() ended on the last in_slots() test, so under set -e/pipefail a
SLOTS-narrowed run exited right after its first parity loop: the runs had
happened, but the failure warning and the rest of the script never ran.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…ir may sit outside the repo (lane BENCH1, step 2)

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…uota, one rejection replaced (lane BENCH1, staged step 1)

pineforge-utils/bench-maintenance/cloud_compile_ratelimited.py (a wrapper
around cloud_compile.py's pyne compile command) sent 203 compile requests
and 1 usage probe in three batches: 13:36-13:44Z (1 + 89), 14:36-14:44Z
(90), 15:36-15:40Z (24) -- never more than 90 in any rolling hour, one at a
time 5 s apart, zero HTTP 429. Every request is in
results/staged/pynesys-compile-log.md (timestamps and outcomes; no key).

199 of the 200 selected slots compiled (PyneComp v6.0.68). Slot 192's
source carries //@Version=6 then two //@Version=5 lines and PyneSys
rejects it ("Empty document."); the slot is kept -- its PyneCore cell reads
n/a with that error -- and select_population.py --replace appends the
first member of its bin's queue as slot 201 (hard surface, bin 21, 154 TV
trades), compiled ok. The canonical indicator script was recompiled too.
The selection manifest moves to results/staged/ with the staged results
and regenerates byte-identically from its inputs and the recorded
rejection.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

… against TradingView by the canonical rubric (lane BENCH1, staged step 2)

run_all.sh (SKIP_BUILD/PINEFORGE/PINETS/SPEED/REPORTS/VECTORBT, JOBS=8,
SLOTS chunks) ran runners/run_pynecore.py (PyneCore 6.10.2) on all 201
slots of both roots; compare.py --engines PyneCore graded every
pynecore_trades.csv against its tape with verify_corpus.analyze_strategy
into results/staged/pynecore_summary.md (+ per-row metrics in
pynecore_trade_comparison.md); the PineForge column reads "pending wave D
(BENCH2)" and vectorbt "runs in BENCH2". compare.py's summary now lists
every non-excellent row with its failing gates.

PyneCore vs TV: corpus 100 = 84 excellent / 12 strong / 3 weak /
1 minimal; closed 101 = 49 excellent / 30 strong / 8 moderate / 9 weak /
1 minimal / 4 n/a (192 PyneSys compile rejection; 140, 166, 197 PyneCore
security-context RuntimeError on the feed's first, partial daily period).
Trades emitted 371,598 over the whole feed vs 266,451 TV trades.

The canonical indicators: PineTS 0.9.34 reproduces the committed
canonical_pinets.csv byte-for-byte; PyneCore (recompiled canonical_pyne.py)
now covers the 53,929-bar feed; PyneCore <-> PineTS max relative delta
3.45e-07 (results/staged/indicator_comparison_pynecore_pinets.md). The
PineForge pairs are pending wave D.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

… 4794c69 (lane BENCH1, staged step 5)

benchmarks/assets now points at pineforge-4pass/pineforge-benchmarks-assets
refs/heads/r5/BENCH1 = 4794c69f4ebe63c84d2e6878873b30db8b3468d2 (pushed):
the 100 corpus slots 001-100 with strategy.pine, tv_trades.csv, inputs.json,
generated.cpp, strategy_pyne.py and pynecore_trades.csv, and the refreshed
PyneCore canonical indicator files. The previous gitlink, bb97c51 (the
retired 100-slot suite), stays reachable on the assets main branch.

The 101 closed slots (100 selected + replacement 201) are not public: their
tarball is in the maintainers' evidence store as
sha256/6e938f9a9160eeae6bda5c7c2d02078f1b9795d767a48d125fa68245f69ca539
(14,696,398 bytes; lab evidence get <sha> --out <path>).

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…et in four hours; the staged README (lane BENCH1, staged steps 3-4)

results/staged/pynecore_speed.md: the PyneCore timing gate (1-min load
< 6.0, no cmake --build / ctest / ci_verify process) was polled every
~5 minutes from 2026-09-21 15:49:00Z to 19:49:13Z -- 100 probes, none
passed (1-min load min 7.74 with 8 build/test processes, median 97.0,
max 233.2; 5-16 such processes at every probe, other lanes' ci_verify
runs). No timing was taken and no pynecore_speed.json is staged;
quiet_host_polls.tsv holds all 120 probes of the lane.

results/staged/README.md: what is staged and where (assets gitlink
4794c69, closed tarball sha256 6e938f9a...), the PyneSys request counts,
the PyneCore tier table and its mechanisms, and the exact commands lane
BENCH2 runs for the PineForge half (and the PyneCore timing) from cold.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…us 100/100 excellent, closed 100 excellent + 1 strong (lane BENCH2, step 3)

Engine main e9ad37d runs the committed generated.cpp (codegen 89645d6,
unchanged since BENCH1) on the 53,929-bar bench feed; compare.py grades
PineForge, PyneCore (the staged outputs) and vectorbt against the TV
tapes with the canonical verify_corpus.analyze_strategy rubric.

- PineForge: corpus 100 excellent; closed (101 slots) 100 excellent +
  1 strong (181-roi10x-shiva-lt-ls-blend, one extra trade at the window's
  opening bars, count delta 1 of 2,411).
- PyneCore 6.10.2: 133/42/8/12/2 + 4 n/a, byte-identical to the staged
  BENCH1 table.
- vectorbt 0.28.2: 13 of the 14 carried-over ports load (4/6/2/1).
- indicator_comparison.md: 53,909 common bars; PF<->PC p99 <= 4.3e-10,
  max-rel carries the documented EMA-seeding warmup divergence.

benchmarks/assets 4794c69 -> 6aedbcf (assets r5/BENCH1, pushed): the 100
public pineforge_trades.csv, 13 vectorbt_trades.csv, the canonical
PineForge indicator CSV, and the assets README.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…yneCore 36.6k, median 15x; throughput package re-measured (lane BENCH2, step 4)

All batches gated on 1-min load < 6 with no cmake --build / ctest /
ci_verify process (by executable name); every batch passed at load
3.79-5.93 with zero such processes, recorded per batch in speed.md.

- PineForge GBench /throughput/with_magnifier, 201 slots: median
  88.31 ms, bars/s Q1 433.7k / median 610.7k / Q3 725.8k.
- PyneCore time_pynecore.py --n 20 --workers 8 in slot chunks: 196 of
  201 timed (192 no source; 140/166/197 RuntimeError; 143 the same error,
  intermittent), median 1,475 ms, 36.6k bars/s; slots 162 and 178
  outlived a chunk and were re-timed with time_one's own loop.
- Median speedup PineForge vs PyneCore 15x (p5 6x, p95 70x, n=196);
  vs vectorbt 1.3x (n=13); PineTS canonical 485.8 ms.
- throughput/reproduce.sh x5 (no-magnifier hot loop, N=201): run
  medians 0.604/0.615/0.606/0.620/0.614 M bars/s; the median run (5)
  is kept as benchmark_results.json + throughput_quartiles.png.

aggregate.py gains --engine-label and --provenance (speed.md's closing
section: the BENCH1 "never quiet" record, every deviation, and the
same-host check: engine ac011d8 still times its June figures on this
host, e9ad37d is 12-20x slower per bar on the three shared probes).
grid_search_repro.py sweeps 021-composite-scalping-integration-01 (the
retired 19-scalping-wunder-bots is gone), plot_quartile.py titles the
real N, and reproduce.sh now builds the GBench harness it runs.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…e to results/, the staged PyneCore reports are superseded (lane BENCH2, step 5)

selection.{json,md} and pynesys-compile-log.md move to results/ (and
select_population.py writes there by default); pynecore_summary.md,
pynecore_trade_comparison.md and indicator_comparison_pynecore_pinets.md
are superseded by results/{summary,trade_comparison,indicator_comparison}.md,
pynecore_speed.md's "not measured" record and quiet_host_polls.tsv fold
into speed.md's provenance, and the staged README's cold-start recipe
becomes benchmarks/README.md's "Reproduce" section.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

… per group, the PyneCore mechanisms, median 15x vs PyneCore (lane BENCH2, step 5)

README.md: the Speed badge and the "Fast." bullet (median 15x on 196
timed strategies), and the Cross-engine comparison section rewritten
for the new population (100 corpus probes + 100 closed ETHUSDT.P/15
scraped scripts whose artifacts stay in the maintainers' evidence
store, sha 6e938f9a...), the tier table per engine per group, the
PyneCore paragraph (feed-window carry, request.security, grid and
str.match rows; 1 compile rejection, 3 runtime errors), the speed and
throughput figures, why neither is comparable to the 2026-06-11 table,
and the Last refresh line. benchmarks/README.md and the throughput
package README likewise, with the Reproduce recipe.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…oot bench lines (lane BENCH2, step 7)

BENCH1's five closed-root lines in the bench-strategies block shift
`if(PINEFORGE_BUILD_LIVE_RUNNER)` from 422 to 427;
check_doc_anchors.py reports the stale CMakeLists.txt:422-423 citation
(ambiguous, so not auto-fixed). One line.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…forge.a is newer (lane BENCH3)

The runner links libpineforge.a statically, but run_all.sh rebuilt it only when
run_pineforge_canonical.cpp changed. After an engine change the indicator column
came from a binary linked against an old library (the committed one dated from
2026-09-21, before wave D). The rebuilt runner reproduces canonical_pineforge.csv
byte for byte at 063e446.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

…201 trade lists unchanged, 89 ms / 603k bars/s, 15x vs PyneCore (lane BENCH3)

Parity: every dylib and the GBench harness rebuilt on main 063e446; all 201
pineforge_trades.csv are byte-identical to the e9ad37d refresh, so summary.md,
trade_comparison.md and indicator_comparison.md regenerate unchanged (corpus
100/100 excellent, closed 100 excellent + 1 strong, slot 181 unchanged).

Speed: PineForge re-timed in two GBench batches (slots 001-100 at load 5.41,
101-201 at 5.90): median 89.45 ms, 603k bars/s (Q1 425k, Q3 731k), median
1.02x the e9ad37d time per strategy. PyneCore, PineTS and vectorbt are not
re-measured; their timing files reproduce the e9ad37d report byte for byte.
The throughput package is not re-timed: the host was never quiet.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

… over five quiet runs; June-engine A/B re-measured beside e9ad37d (lane BENCH3)

Throughput: five gate-checked reproduce.sh runs (run medians 0.621, 0.638,
0.611, 0.646, 0.640 M bars/s); the package publishes run 2, the median run
(e9ad37d measured 0.614). Two runs were discarded: one saw another project's
load spike to 16.8 before its benchmark began, one recompiled the bench tree
inside the run after a commit moved the version header.

A/B, one window, alternating: 063e446 takes 0.94x the time of e9ad37d per
strategy on 26 public slots (22 of 26 faster), and 10-18x the June engine
ac011d8 on the three probes both populations share (e9ad37d: 11-19x). The
published PineForge column stays the sweep at load 5.41/5.90; its cross-window
comparison with the e9ad37d sweep (1.02x) is the weaker one and is labelled so.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>