Repository navigation
[pull] main from pineforge-4pass:main - #31
Merged
Merged
Conversation
hour()/minute()/second()/dayof*/month()/year()/weekofyear() were codegen'd
as an inline setenv("TZ")+tzset()+localtime_r lambda evaluated once per bar.
On macOS every changed-TZ tzset() does a notifyd Mach-IPC round-trip (~173us);
across a full ETH 1m feed (~888k calls/field) this cost minutes of wall time
that read as an engine "hang" (algotorma: 154s+ -> never finished).
Add churn-free pine_hour()/pine_minute()/... helpers backed by
decompose_ms_local(): UTC takes the tzset-free gmtime_r path, other zones use
the cached pine_tz::ScopedTimezone (skips tzset when the active zone is
unchanged, never restores) so a single-zone run pays one tzset total. Also
short-circuit calendar_day_open_local_ms() for UTC (integer midnight floor) so
the per-bar tradingday path stops flipping the process TZ to UTC and back,
which would otherwise re-slow the strategy's own zone.
Value-identical to the old lambda: same Pine offsets (dayofweek=tm_wday+1,
month=tm_mon+1, year=tm_year+1900, weekofyear ISO-ish), DST-exact via
localtime_r over the tz database. Verified NY summer h=8 EDT / winter h=7 EST,
Phoenix unshifted, Kolkata +5:30; lukeborgerding trades byte-identical to
baseline; corpus gate 252/252 no demotions; ctest 79/79; 16 formerly-"hung"
strategies now finish in seconds.
Codegen emits calls to these helpers in a paired pineforge-codegen-oss change.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
fix(session_time): kill per-call tzset()->notifyd storm on macOS (KI-35)
_load_bars parsed the CSV and built the BarC[] with a per-bar Python/ctypes loop that was ~99% of a run's wall time (the C++ backtest itself is ~20ms at ~27M bar/s). numpy parses the whole feed at once and BarC[] shares its exact memory layout (5x float64 + 1x int64, no padding), so a single from_buffer_copy builds the array with zero per-bar Python work — 1m feed 4.93s -> 0.62s, byte-identical output. Falls back to the explicit loop when numpy is absent. Also add Strategy.run(preloaded_bars=(BarC[], n)) so a batch driver can parse the shared feed ONCE and reuse it across every strategy/candidate, skipping the CSV parse + ctypes build entirely per run. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…zed-loader perf(run_strategy): vectorized bar loader + preloaded_bars batch reuse
…KI-35 follow-up) The KI-35 fix (PR #70) passed an ALREADY-normalized tz string into pine_tz::ScopedTimezone, which normalizes again internally (normalize_timezone_for_posix, timezone.cpp). For fixed-offset tz inputs the normalizer flips the POSIX sign ("UTC+2" -> "UTC-2"), so normalizing twice flips it back ("UTC+2" -> "UTC-2" -> "UTC+2") and localtime_r then decodes the wrong wall-clock hour. IANA names and "UTC" are idempotent under normalize, so the bug was invisible to the corpus gate (0 offset-tz strategies), the lukeborgerding trade diff, and the NY/Phoenix/Kolkata DST unit test — every R7 probe used UTC or an IANA zone. Fix: detect UTC on the normalized string (as before) but hand the RAW tz to ScopedTimezone so it normalizes exactly once, matching the old inline lambda. Value-identity restored for offset zones: pine_hour/pine_dayofweek now equal the old setenv+tzset+localtime_r lambda for UTC±N / GMT±N as well as UTC/IANA (standalone A/B over 5 timestamps × {UTC+2,+3,+5,GMT+2,UTC}: all match; NY DST 8 EDT / 7 EST, Phoenix 5/5, Kolkata 17:30 unchanged). ctest 79/79. Blast radius = 5 data/draft strategies with offset-tz session logic (0 corpus). Re-verified all 5 with the fixed lib: mylivingedge-gold-asian-range minimal->excellent (its Asian window was mis-shifted by the double-flip), ananthanvelmurugan/evrinb7716/ltony excellent (held), andrewwieiw moderate (unchanged). Score 323->324, 0 minimal. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…-normalize fix(session_time): stop double-normalizing tz for fixed offsets (KI-35 follow-up)
…tion (KI-31)
process_margin_call computed the forced-liquidation lot as
qty_liq = floor(4 * (qty - equity/(adverse*pv*m)) / step) * step
i.e. it applied the 4x over-liquidation to the RAW minimum-restore qty and only
floored the product. TradingView floors the minimum-restore qty to the
instrument lot step FIRST, then multiplies by 4. Flooring after injects a
~step/4 error into the first nibble that compounds ~3x per step through the
margin-call cascade.
Two coupled corrections:
1. floor q_min to qty_step_ before `qty_liq = 4 * q_min`.
2. the (now redundant) second floor of 4*q_min uses the engine's standard
+1e-6 epsilon (as quantize_qty does): 4*(a lot multiple) is mathematically
a lot multiple, but binary float makes e.g. 4*5.7089 = 22.83559999… which a
bare std::floor drops a whole lot. Without the epsilon the tail nibbles
desync from ~step 14 on.
Row-diff vs the ETHUSDT.P export (alpha-wizard-channel percent_of_equity=100,
cascade-1): 19/19 margin-call nibbles now bit-exact (7.7232/30.3796/35.716/…);
the old floor-after matched 0/19 and desynced by step 7.
Blast radius: qty_step_ == 0 for the entire corpus (only verify-engine-local
passes 0.0001 for the ETHUSDT.P scraper feed); the explicit-leverage p2/5x
margin probes never set it. So corpus + those probes are BYTE-IDENTICAL — this
only sharpens the ETHUSDT.P percent_of_equity margin-cascade nibbles. Note: it
does NOT change any scraper strategy's tier (the ~40 margin-cascade members fail
on entry-coverage / are KI-28 indicator-rewrite provenance-gapped, not on
margin-exit fidelity) — this is a correctness/fidelity fix, score-neutral.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…oor-order fix(margin): floor restore-qty to lot step BEFORE the 4x over-liquidation (KI-31)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )