Skip to content

[pull] main from pineforge-4pass:main - #31

Merged
pull[bot] merged 8 commits into
Malumbo21:mainfrom
pineforge-4pass:main
Jul 7, 2026
Merged

pull[bot] merged 8 commits into
Malumbo21:mainfrom
pineforge-4pass:main

Conversation

@pull

@pull pull Bot commented Jul 7, 2026 •

Copy link
Copy Markdown

See Commits and Changes for more details.


Created by pull[bot] (v2.0.0-alpha.4)

Can you help keep this open source service alive? 💖 Please sponsor : )

luisleo526 and others added 8 commits July 7, 2026 12:49
hour()/minute()/second()/dayof*/month()/year()/weekofyear() were codegen'd
as an inline setenv("TZ")+tzset()+localtime_r lambda evaluated once per bar.
On macOS every changed-TZ tzset() does a notifyd Mach-IPC round-trip (~173us);
across a full ETH 1m feed (~888k calls/field) this cost minutes of wall time
that read as an engine "hang" (algotorma: 154s+ -> never finished).

Add churn-free pine_hour()/pine_minute()/... helpers backed by
decompose_ms_local(): UTC takes the tzset-free gmtime_r path, other zones use
the cached pine_tz::ScopedTimezone (skips tzset when the active zone is
unchanged, never restores) so a single-zone run pays one tzset total. Also
short-circuit calendar_day_open_local_ms() for UTC (integer midnight floor) so
the per-bar tradingday path stops flipping the process TZ to UTC and back,
which would otherwise re-slow the strategy's own zone.

Value-identical to the old lambda: same Pine offsets (dayofweek=tm_wday+1,
month=tm_mon+1, year=tm_year+1900, weekofyear ISO-ish), DST-exact via
localtime_r over the tz database. Verified NY summer h=8 EDT / winter h=7 EST,
Phoenix unshifted, Kolkata +5:30; lukeborgerding trades byte-identical to
baseline; corpus gate 252/252 no demotions; ctest 79/79; 16 formerly-"hung"
strategies now finish in seconds.

Codegen emits calls to these helpers in a paired pineforge-codegen-oss change.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
fix(session_time): kill per-call tzset()->notifyd storm on macOS (KI-35)
_load_bars parsed the CSV and built the BarC[] with a per-bar Python/ctypes
loop that was ~99% of a run's wall time (the C++ backtest itself is ~20ms at
~27M bar/s). numpy parses the whole feed at once and BarC[] shares its exact
memory layout (5x float64 + 1x int64, no padding), so a single from_buffer_copy
builds the array with zero per-bar Python work — 1m feed 4.93s -> 0.62s,
byte-identical output. Falls back to the explicit loop when numpy is absent.

Also add Strategy.run(preloaded_bars=(BarC[], n)) so a batch driver can parse
the shared feed ONCE and reuse it across every strategy/candidate, skipping the
CSV parse + ctypes build entirely per run.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…zed-loader

perf(run_strategy): vectorized bar loader + preloaded_bars batch reuse
…KI-35 follow-up)

The KI-35 fix (PR #70) passed an ALREADY-normalized tz string into
pine_tz::ScopedTimezone, which normalizes again internally
(normalize_timezone_for_posix, timezone.cpp). For fixed-offset tz inputs the
normalizer flips the POSIX sign ("UTC+2" -> "UTC-2"), so normalizing twice
flips it back ("UTC+2" -> "UTC-2" -> "UTC+2") and localtime_r then decodes the
wrong wall-clock hour. IANA names and "UTC" are idempotent under normalize, so
the bug was invisible to the corpus gate (0 offset-tz strategies), the
lukeborgerding trade diff, and the NY/Phoenix/Kolkata DST unit test — every R7
probe used UTC or an IANA zone.

Fix: detect UTC on the normalized string (as before) but hand the RAW tz to
ScopedTimezone so it normalizes exactly once, matching the old inline lambda.

Value-identity restored for offset zones: pine_hour/pine_dayofweek now equal
the old setenv+tzset+localtime_r lambda for UTC±N / GMT±N as well as UTC/IANA
(standalone A/B over 5 timestamps × {UTC+2,+3,+5,GMT+2,UTC}: all match; NY DST
8 EDT / 7 EST, Phoenix 5/5, Kolkata 17:30 unchanged). ctest 79/79.

Blast radius = 5 data/draft strategies with offset-tz session logic (0 corpus).
Re-verified all 5 with the fixed lib: mylivingedge-gold-asian-range
minimal->excellent (its Asian window was mis-shifted by the double-flip),
ananthanvelmurugan/evrinb7716/ltony excellent (held), andrewwieiw moderate
(unchanged). Score 323->324, 0 minimal.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…-normalize

fix(session_time): stop double-normalizing tz for fixed offsets (KI-35 follow-up)
…tion (KI-31)

process_margin_call computed the forced-liquidation lot as
  qty_liq = floor(4 * (qty - equity/(adverse*pv*m)) / step) * step
i.e. it applied the 4x over-liquidation to the RAW minimum-restore qty and only
floored the product. TradingView floors the minimum-restore qty to the
instrument lot step FIRST, then multiplies by 4. Flooring after injects a
~step/4 error into the first nibble that compounds ~3x per step through the
margin-call cascade.

Two coupled corrections:
  1. floor q_min to qty_step_ before `qty_liq = 4 * q_min`.
  2. the (now redundant) second floor of 4*q_min uses the engine's standard
     +1e-6 epsilon (as quantize_qty does): 4*(a lot multiple) is mathematically
     a lot multiple, but binary float makes e.g. 4*5.7089 = 22.83559999… which a
     bare std::floor drops a whole lot. Without the epsilon the tail nibbles
     desync from ~step 14 on.

Row-diff vs the ETHUSDT.P export (alpha-wizard-channel percent_of_equity=100,
cascade-1): 19/19 margin-call nibbles now bit-exact (7.7232/30.3796/35.716/…);
the old floor-after matched 0/19 and desynced by step 7.

Blast radius: qty_step_ == 0 for the entire corpus (only verify-engine-local
passes 0.0001 for the ETHUSDT.P scraper feed); the explicit-leverage p2/5x
margin probes never set it. So corpus + those probes are BYTE-IDENTICAL — this
only sharpens the ETHUSDT.P percent_of_equity margin-cascade nibbles. Note: it
does NOT change any scraper strategy's tier (the ~40 margin-cascade members fail
on entry-coverage / are KI-28 indicator-rewrite provenance-gapped, not on
margin-exit fidelity) — this is a correctness/fidelity fix, score-neutral.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…oor-order

fix(margin): floor restore-qty to lot step BEFORE the 4x over-liquidation (KI-31)
@pull pull Bot locked and limited conversation to collaborators Jul 7, 2026
@pull pull Bot added the ⤵️ pull label Jul 7, 2026
@pull
pull Bot merged commit 15b02e0 into Malumbo21:main Jul 7, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant