Skip to content

Pine adapter: recover the per-bar cost of the margin, sizing and admission rules - #336

Merged
luisleo526 merged 3 commits into
mainfrom
px/perf-n2
Oct 4, 2026
Merged

luisleo526 merged 3 commits into
mainfrom
px/perf-n2

Conversation

@luisleo526

@luisleo526 luisleo526 commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

What

The margin, sizing and admission rules that landed in #330 and #335 made each bar more expensive, and this PR recovers that cost. The change is performance only: no decision, order of operations, hashed field or floating-point value moves. It has three commits:

  1. The retired-row sweep reads the placement rows in place. On a gathered chart (script timeframe above the input timeframe) the sweep at each bar open walked a table that grows with the run. It now reads the rows straight off storage in the same order, and tests a row's own pin before the searches.
  2. The margin and admission scopes answer from the configuration. Their configuration-only conjuncts are formed when the configuration is set; set_configuration and set_staged_configuration are the only writers. A one-command same-bar queue is no longer sorted, two helpers decide before copying a row, and the realized-money fold returns at once when nothing was booked.
  3. Tests: the lookup-index and live-state scaling rows run serially in Release. Under a parallel ctest the lookup-index ratio once read 6.15 against its 5.0 bound, and 4.08 when run alone. Test properties only.

Value identity

Each commit is checked against its parent:

  • every pinned state hash (30 witness harvests, no re-pin);
  • a whole-suite dump of every closed and range-end trade, with all 21 Trade fields at %.17g and every recorded per-bar, broker and stream state hash. It is byte-identical except rows that differ by run, not by tree; each was confirmed by running it alone, twice per tree;
  • the no-hash digests and all tape and witness executables, byte-identical;
  • release ctest with the receipt rows;
  • the corpus, 312/312 with 0 moved.

An independent review ran the old scope predicates beside the new ones over the full suite: 0 differences and 0 stale cached flags in 30.35 M re-derivations, and 8 M generated configurations pass.

Timing

Measured on a Xeon 8375C, with 4 independent builds per tree resampled as a random effect and paired blocks. The table shows the whole-batch median, percent against 14b05c9 (before #330), with 95% intervals.

example script TF = input TF 1-minute input, 3-minute script
native example −1.5 [−1.5, −1.4] −5.2 [−5.3, −5.0]
ta-sma-dual-cross −1.9 [−2.0, −1.9] −2.2 [−2.3, −2.1]
bracket-aurora +0.2 [+0.1, +0.2] −3.6 [−3.7, −3.5]
udt-regime-stack +0.4 [+0.4, +0.5] −2.6 [−2.8, −2.5]
composite-vcp +2.6 [+2.5, +2.6] −0.4 [−0.5, −0.2] †

Before this PR, main stood at +6.7, +2.9, +1.4, +5.2 and +4.4 on the first column, and +4.5, +2.6, +1.1, +3.4 and +2.0 on the second. Differences under 0.5 % on the whole-batch median read as at par.

Verification

  • Engine verify bundles (preflight, release, sanitizers, docs, debug, kernel, parity, subset) run on this exact tree.
  • One parity sweep of the full population runs on this head, paired with codegen main.

🤖 Generated with Claude Code

luisleo526 and others added 3 commits October 5, 2026 06:33
On a gathered chart (a script timeframe above the input timeframe),
pin K5 keeps every entry row of the run, so the sweep at each bar open
(erase_retired_rows) walked a table that grows with the run,
re-finding each row and binary-searching the askable origins before
reading the row's own pin: the per-bar cost rose with every entry
placed. The sweep now reads the rows straight off the table's storage,
in the same incarnation order (PlacementTable::visit_rows), and tests
a row's own pin before the searches. It keeps and erases exactly the
rows it did.

Alongside, each value-identical: decimal_grid_exponent forms its
nineteen powers once, by the same expressions; gain_loss_regime and
slipped_long_margin_scope read the configuration before the run;
on_bar_open takes the point-order scan only under the point order; the
narrow-scope margin and admission helpers stay out of line (two) or
out of line and cold (six), and thirteen of their scope tests are
hinted rare.

The recovery is for gathered charts. With the script timeframe equal to
the input timeframe the table stays small, the sweep is cheap anyway,
and the cost the margin, sizing and admission rules added since
14b05c9 remains (native example batch median +5.7 %); the next
commit takes it.

No decision, order of operations, hashed field or floating-point value
moves: against the admission change (#335) the witness harvests (every
pinned state hash), a per-trade dump of all 21 Trade fields (%.17g,
every run pass), the no-hash digests and the tape suites are
byte-identical, and release ctest passes with the receipt rows. Doc
citation digits re-anchored.

Timing, measured on the admission change (#335) before its final rebase
(same library):
Xeon 8375C, CPU 2, 61 rotating blocks x 3 independent builds per tree
x 5 runs, paired block bootstrap; every 95% interval within 0.21
points, * = it covers 0.

  percent vs                native  ta-sma  bracket    udt    vcp
  batch median, gathered (1 m input, 3 m script)
    14b05c9                +0.05*  -0.05*   -1.95   -0.38  -0.29
    #335 (parent)           -4.31   -2.56    -2.98   -3.63  -2.07
  batch median, ungathered (1 m input, 1 m script)
    14b05c9                +5.70   -0.33    +0.07*  +4.19  +2.09
    #335 (parent)           -1.06   -3.01    -1.24   -1.00  -2.25
  stream p95, gathered
    14b05c9                +0.89   -0.18    -0.90   +2.77  +3.41

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…on; no sort or copy for a lone command

The margin and admission scopes are asked several times a bar, and on a
run outside them -- no power-of-ten lot step, a commission, pyramiding
above one -- the configuration alone answers. Those conjuncts are now
formed when the configuration is set (set_configuration and
set_staged_configuration, the only writers of config_ and staged_), so
such a bar answers each scope with one read instead of a call; inside a
scope its own tests run as before, and the rule switches are still read
at every ask. The unified admission at resolve_terms moves out of line
unchanged, and the realized-money fold returns at once when nothing was
booked since the last one.

Alongside, on the same per-bar path: the same-bar command flush no longer
stable-sorts a one-command queue (std::stable_sort still took a
temporary buffer and moved the command, its whole placement snapshot
included, through it and back) and keeps the queue's storage for the
next bar; cancel_bracket_siblings and apply_fx_opening_margin_slice
decide before copying the filled row; close_all records the close-first
fact from the point and position it has already read; on_bar_open forms
the opening print only under the point order; the retired-row sweep's
visitors are inlined.

No decision, order of operations, hashed field or floating-point
value moves. Against the previous commit: the whole-suite dump
(every closed and range-end trade with all 21 Trade fields at %.17g,
every recorded per-bar, broker and stream state hash) is
byte-identical on 659 of 663 process files (4 of them matched by
content: their name hashes a temp-path argument), and the other 4
differ by run, not by tree (a timing-calibrated row, a stack-address
row, two concurrent writers identical when run alone, twice in each
tree); the 30 witness harvests (no re-pin) and 176 tape and witness
executables are byte-identical; release ci_verify 859/859 with the
receipt rows; corpus 312/312, moved 0. The seven new members are
functions of the folded configuration and are waived as such. Doc
citation digits re-anchored.

Timing (Xeon 8375C, CPU 2; 4 independent builds per tree resampled as a
random effect, blocks paired; percent against 14b05c9, 95% CI; largest
build-to-build A/A 0.11 % / 0.27 % on the whole-batch median):

                        whole-batch median  stream mean per call  stream p95
script timeframe = input timeframe (81 blocks)
  native example        -1.5 [-1.5,-1.4]    -2.8 [-2.8,-2.7]      -2.1 [-2.2,-2.1]
  ta-sma-dual-cross     -1.9 [-2.0,-1.9]    -0.5 [-0.5,-0.4]      +0.4 [+0.4,+0.5]
  bracket-aurora        +0.2 [+0.1,+0.2]    -0.7 [-0.8,-0.7]      +1.0 [+1.0,+1.1]
  udt-regime-stack      +0.4 [+0.4,+0.5]    -0.8 [-0.9,-0.8]      +0.4 [+0.3,+0.5]
  composite-vcp         +2.6 [+2.5,+2.6]    +1.6 [+1.5,+1.6]      +2.2 [+2.1,+2.3]
1-minute input under a 3-minute script (41 blocks)
  native example        -5.2 [-5.3,-5.0]    -4.4 [-4.5,-4.3]      -5.0 [-5.1,-4.9]
  ta-sma-dual-cross     -2.2 [-2.3,-2.1]    -1.2 [-1.3,-1.1]      +0.2 [+0.1,+0.5]
  bracket-aurora        -3.6 [-3.7,-3.5]    -3.3 [-3.4,-3.2]      -5.8 [-5.9,-5.7]
  udt-regime-stack      -2.6 [-2.8,-2.5]    -2.3 [-2.4,-2.2]      -0.4 [-0.7,-0.1]
  composite-vcp †       -0.4 [-0.5,-0.2]    +0.4 [+0.3,+0.4]      +2.0 [+2.0,+2.2]

† Not a property of this commit: this row moves with the size of the
process environment. setenv("TZ") (src/timezone.cpp) costs one heap
trim and re-grow per run, about 85 us over 5,003 bars, and the row
reads -0.3 to -0.6 with some environment sizes and +1.5 (+1.9 against
the previous commit) with others. Whole-batch differences under
0.5 % read as at par.

In the same runs the previous commit stood at +5.6, -0.3, +0.1, +4.2,
+2.0 (script timeframe = input timeframe) and -0.2, -0.1, -2.0, -0.4,
-0.1 (3-minute script) on the whole-batch median, and the admission
change (#335) at +6.7, +2.9, +1.4, +5.2, +4.4 and +4.5, +2.6, +1.1,
+3.4, +2.0. composite-vcp's remaining gap is the session clock in
src/session_time.cpp (time() with a session argument every bar), which
this commit does not touch.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…elease

Under a parallel ctest the lookup-index scaling row once read 6.15
against its 5.0 bound, and 4.08 when run alone; the live-state scaling
row failed the same way under load. RUN_SERIAL keeps other rows off the
box while these two time their legs, in Release, where their ratios
gate. A whole Release suite at -j 28 read 3.99-4.01 with it and
3.82-4.83 without, for about a minute more wall time. No source or
behaviour change.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@luisleo526
luisleo526 marked this pull request as ready for review October 4, 2026 23:28
@luisleo526
luisleo526 merged commit a80c623 into main Oct 4, 2026
17 checks passed
@luisleo526
luisleo526 deleted the px/perf-n2 branch October 4, 2026 23:29
luisleo526 added a commit that referenced this pull request Oct 5, 2026
…ents resolved without copies (#339)

## What

`time(tf, session, tz)` with a script's own session argument reads TradingView's session clock outside UTC (#334). That path asked the per-day session cache about nine times a bar, and copied the session and zone strings on every `time()` / `time_close()` call. This PR reads each day's occurrences once per call, caches `utc_zone` per session-cache entry, and resolves the call's strings by reference. It changes one file, `src/session_time.cpp`: the language calendar functions under rule 1 of ADR 0001. It adds no switch, input, API or hashed state.

## Value identity (on this tree, against main)

- the old and new session functions run side by side over the whole suite: 0 mismatches;
- a differential over generated sessions, zones and timeframes, walking every DST switch of 2024–2026 minute by minute: 16.6 M queries, identical digest;
- an independent review repeated it with 33.2 M queries: 0 different answers, identical cache and zone-memo history, clean under ASan + UBSan;
- the 30 witness harvests: byte-identical, 0 re-pins;
- the whole-suite per-trade dump (all 21 Trade fields at `%.17g`, every state hash): identical apart from rows that differ between two runs of one tree;
- all 307 tape and witness executables: byte-identical;
- corpus 312/312, 0 moved.

## Timing (on this tree, against main 44eab7b)

Measured on a Xeon 8375C with 4 independent builds per tree, paired rotating blocks (81 per shape), and every process under exactly 22 environment variables. Positive means slower; `*` means the 95% interval covers 0. The **padded control** is this commit with every function outside `session_time.o` placed at main's address, checked symbol by symbol.

| | native | ta-sma | bracket | udt | composite-vcp |
|---|---|---|---|---|---|
| batch median, ungathered (1 m / 1 m) | +1.23 | +1.26 | +0.87 | +0.83 | **−9.54** |
| — padded control | +0.03* | +0.06* | +0.02* | −0.00* | **−7.93** |
| per call, ungathered | +0.21 | +0.04* | +0.19 | +0.10 | **−5.16** |
| — padded control | +0.05* | +0.01* | −0.02* | −0.02* | **−4.84** |
| stream p95, ungathered | +0.46 | +1.53 | +0.46 | +0.17 | **−3.07** |
| — padded control | +0.04* | +0.03* | −0.05* | +0.01* | **−3.46** |
| batch median, gathered (1 m / 3 m) | +0.60 | +0.09* | +0.54 | +0.41 | **−5.80** |
| — padded control | +0.06* | +0.06* | +0.05* | +0.09* | **−5.25** |

- **Only composite-vcp's work changes.** Its instruction count drops from 232.5 M to 220.1 M over 2 × 5,000 stream pushes, and from 2,512 to 1,275 per `timeframe_time` call.
- **The other four cases execute identical instructions.** Their differences against main are code placement: this file shrinks by 263 bytes and moves the hot functions by 192–256 bytes. With the layout held fixed (padded control), all four read 0 within noise on every metric. The commit message's table was measured with #336's tree as the base; the table above is this tree.

## Verification

- Engine verify bundles on this exact tree: release 859, sanitizers 837, debug 840, kernel 307 with 0 boundary findings, parity 312/312, subset, docs.
- One parity sweep of the full population on this head, paired with codegen main, is neutral: 7,989 comparable, no moves.
- An independent review returned MERGE; its message fixes are applied.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
luisleo526 added a commit that referenced this pull request Oct 5, 2026
* docs: release notes and version scoping for v1.2.0

CHANGELOG: the Unreleased section becomes 1.2.0 (2026-10-05), a minor
release that adds the compiled execution-capabilities extension to the C
ABI (PF_CAPABILITIES_API_VERSION, strategy_capabilities_api_version,
strategy_capabilities_receipt). One bullet per change since v1.1.0 with
its PR (#328, #330, #331, #332, #333, #334, #335, #336, #337, #338 and the
session-clock change, written #VCP until its PR number exists), the
recipe additions under State hashes (#328, #330, #335; domain tags and
epoch unchanged), Report keys (docker/run_json.py is byte-identical to
v1.1.0's, so no key changes) and Migration.

Version scoping: README, install and public-contract pages name v1.2.0
and codegen 1.2.0; "since the next release" and "fixed on main" in the
capabilities guide and the runner guide become 1.2.0 and 1.1.0 (#325).
Lines that state what an earlier release shipped stay.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs: 1.2.0's grades from the facts tokens, #339, and codegen's license

- CHANGELOG: the session-clock change is #339 (merged as 52292db), in place
  of the #VCP token of 3082fbd; the "What a script author sees" bullet of
  3082fbd now states release 1.2.0's grades (registry baseline
  pineforge-parity-baseline-20261005-engine-52292db9: 7,970 excellent / 19
  strong of 7,989, none below strong) instead of the main scoreboard at
  6b77f06.
- README: the release sentence names 1.2.0 and renders releases[1.2.0], and
  the main scoreboard renders the same baseline, from pineforge-release
  facts/facts.json at 0c2f6838 (the facts export for the 1.2.0 release).
- README, LEGAL.md, docs/pages/index.md: pineforge-codegen is under the
  PineForge Source License 1.1 from its release 1.2.0; releases up to 1.1.0
  keep the PolyForm Noncommercial terms they shipped with.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant