Repository navigation
security: anchor HTF aggregation on the symbol clock (syminfo.timezone + session) - #159
Merged
Merged
Conversation
…e + session) TimeframeAggregator bucketed intraday targets on the raw epoch grid and calendar targets on UTC wall-clock. That is TV-correct only for 24x7 symbols: for session symbols TradingView groups 240m buckets from the session open (09:30 on equities) and D/W/M boundaries at symbol-local midnight, so every session-symbol request.security series was misaligned (AAPL probes: 240m 25.4%, D 56.1%, W 62.0% match). The aggregator now takes the exchange tz + session and keys buckets on (local-midnight + session-open) arithmetic; crosses_boundary/tf_change gain tz-aware forms used by historical projections, streaming replay, and the finer-TF publish gate so all boundary consumers move together. tz=UTC / session empty-or-24x7 reduces to identical integer math: default construction sites pass syminfo_.timezone/.session which stay at those defaults unless runtime_overrides set them, so the ETH corpus regime is bit-for-bit unchanged (133/133 unit tests green).
…+session) Subtracting the session-open offset before the day-floor shifted each session wholesale instead of anchoring it, making D/W/M grouping tz-invariant. Calendar periods now decompose on the exchange clock only; the session offset applies solely to intraday bucket grids.
On session symbols the next-bar-crossing rule never fires inside an RTH day (16:00 close is far from midnight), so D/W/M buckets completed on the NEXT session's first bar — every security series ran one session behind TV (probe-htf-bucket-d: engine lvl on day M was close(M-2) all day where TV refreshes to close(M-1) on M's final bar; 241 vs 180 round trips). When the run declares a same-day session window, a bucket completes on the bar whose close reaches the session end. 24x7/empty/wrapped windows keep the crossing rule bit-for-bit.
The session-close rule fired on every day's final bar, completing weekly buckets mid-week (W probe 62% -> 7.8%). Gate it on the next calendar day starting a new period.
The anchor helpers paid ScopedTimezone+localtime_r+mktime on EVERY call before consulting their day cache — session-symbol runs slowed enough to tip ladder candidates over the sweep timeout. Offset is stable within a UTC hour, so key the memo by hour before touching the slow path.
Hour keys still paid ScopedTimezone(setenv+tzset) ~24x/symbol-day per thread — 50x slowdowns on multi-security strategies over 5y feeds. Offsets change only at DST instants, which fall while session markets are closed, so a per-UTC-day memo is exact for every traded bar.
This was referenced Aug 23, 2026
luisleo526
added a commit
that referenced
this pull request
Oct 12, 2026
Move CODEGEN_COMMIT in scripts/regen_corpus_cpp.sh and the matching literal and message in scripts/test_regen_corpus_cpp_pin.py from codegen a4259656 to the codegen v1.5.0 commit 52101f9b7b540fece4493ddc7495451b6d85fc07. The runtime image pin is unchanged. Measured on one AWS spot box (c5.4xlarge, engine b802cee, corpus b40aa8ec, docker transpile path, the exact edit of this commit applied by the round): - the old-pin control regeneration is clean (314 sources, 0 drifted); - all 314 regenerated generated.cpp change (312 validation probes and the 2 draft probes), 0 transpile failures; - fresh sweep of the old pin versus fresh sweep of the new pin on the same box: the engine trade lists of 312 of 312 validation probes are byte-identical; both headlines read excellent=311 anomaly=1; - the pin test and the VERIFY=1 idempotence check after regeneration pass. The regenerated files are in the paired pineforge-corpus change. Every file gains the same 321 lines, which were introduced by the codegen between the two pins: the coded run stops (codegen #184), the checked strategy settings and C ABI exception containment (#159), and the compiled-execution, confirmed-bar and order-shapes receipts (#167, #173, #178). 6 to 116 further lines per file are probe-specific. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Oct 12, 2026
Move the corpus gitlink from 9245f28a to e99bbd9db257740d9c77db37a8567616627130bf and record it on the `# corpus` line of scripts/corpus_parity_baseline.txt. The 312 baseline digests do not change: the engine trade lists are byte-identical (measured, below). Follows the pin commit of the same pull request. The corpus change (the two commits on top of the removal/headers change) re-emits all 312 validation generated.cpp with codegen 52101f9b7b540fece4493ddc7495451b6d85fc07 and re-derives the 60 generated_cpp_sha256 values of owned_strategies.json; validation_run there still records the 2026-09-26 sweep. Measured on AWS spot boxes: - trades, in the pin round of the previous commit (engine b802cee, corpus b40aa8ec; b802cee..2dbd72f changes only tests and CI scripts, and the 312 generated.cpp compiled there at the new pin are byte-identical to those in e99bbd9d): a fresh full sweep at the old pin and one at the new pin (same box, runtime, harness and feed) give byte-identical engine trade lists for 312 of 312 validation probes; both verifier headlines read excellent=311 anomaly=1; - with this engine (2dbd72f): the bytes in the corpus equal the new-pin regeneration of the final tree on a second box for 312 of 312 files, and VERIFY=1 scripts/regen_corpus_cpp.sh over the final tree reports 312 strategies, 0 drifted, 0 failed to transpile. Every generated.cpp gains the same 321 lines, which were introduced by the codegen between the two pins: the coded run stops (codegen #184), the checked strategy settings and C ABI exception containment (#159), and the compiled-execution, confirmed-bar and order-shapes receipts (#167, #173, #178); 6 to 116 further lines per file are probe-specific. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Oct 13, 2026
…ve the corpus gitlink to e99bbd9 (#377) build: pin the corpus regeneration to codegen 1.5.0 (52101f9b) Move CODEGEN_COMMIT in scripts/regen_corpus_cpp.sh and the matching literal and message in scripts/test_regen_corpus_cpp_pin.py from codegen a4259656 to the codegen v1.5.0 commit 52101f9b7b540fece4493ddc7495451b6d85fc07. The runtime image pin is unchanged. Measured on one AWS spot box (c5.4xlarge, engine b802cee, corpus b40aa8ec, docker transpile path, the exact edit of this commit applied by the round): - the old-pin control regeneration is clean (314 sources, 0 drifted); - all 314 regenerated generated.cpp change (312 validation probes and the 2 draft probes), 0 transpile failures; - fresh sweep of the old pin versus fresh sweep of the new pin on the same box: the engine trade lists of 312 of 312 validation probes are byte-identical; both headlines read excellent=311 anomaly=1; - the pin test and the VERIFY=1 idempotence check after regeneration pass. The regenerated files are in the paired pineforge-corpus change. Every file gains the same 321 lines, which were introduced by the codegen between the two pins: the coded run stops (codegen #184), the checked strategy settings and C ABI exception containment (#159), and the compiled-execution, confirmed-bar and order-shapes receipts (#167, #173, #178). 6 to 116 further lines per file are probe-specific. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> corpus: move the gitlink to e99bbd9d (generated.cpp from codegen 1.5.0) Move the corpus gitlink from 9245f28a to e99bbd9db257740d9c77db37a8567616627130bf and record it on the `# corpus` line of scripts/corpus_parity_baseline.txt. The 312 baseline digests do not change: the engine trade lists are byte-identical (measured, below). Follows the pin commit of the same pull request. The corpus change (the two commits on top of the removal/headers change) re-emits all 312 validation generated.cpp with codegen 52101f9b7b540fece4493ddc7495451b6d85fc07 and re-derives the 60 generated_cpp_sha256 values of owned_strategies.json; validation_run there still records the 2026-09-26 sweep. Measured on AWS spot boxes: - trades, in the pin round of the previous commit (engine b802cee, corpus b40aa8ec; b802cee..2dbd72f changes only tests and CI scripts, and the 312 generated.cpp compiled there at the new pin are byte-identical to those in e99bbd9d): a fresh full sweep at the old pin and one at the new pin (same box, runtime, harness and feed) give byte-identical engine trade lists for 312 of 312 validation probes; both verifier headlines read excellent=311 anomaly=1; - with this engine (2dbd72f): the bytes in the corpus equal the new-pin regeneration of the final tree on a second box for 312 of 312 files, and VERIFY=1 scripts/regen_corpus_cpp.sh over the final tree reports 312 strategies, 0 drifted, 0 failed to transpile. Every generated.cpp gains the same 321 lines, which were introduced by the codegen between the two pins: the coded run stops (codegen #184), the checked strategy settings and C ABI exception containment (#159), and the compiled-execution, confirmed-bar and order-shapes receipts (#167, #173, #178); 6 to 116 further lines per file are probe-specific. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mechanism.
TimeframeAggregatorbucketed intraday targets on the raw epoch grid and calendar targets on UTC wall-clock — TV-correct only for 24x7 symbols. Session symbols anchor differently: TradingView groups intraday HTF buckets from the session open (09:30 equities, 17:00 ET forex) and D/W/M boundaries at symbol-local midnight / session-day. Every session-symbolrequest.securityseries was misaligned (AAPL probes: 240m 25.4%, D 56.1%, W 62.0% match).Fix (generic; per-run config, no strategy-specific paths).
TimeframeAggregator(tz, session): RATIO buckets keyed on(local-midnight + session-open)arithmetic; CALENDAR periods decompose on the exchange clock.crosses_boundary/tf_changetz-aware forms keep historical projections, streaming replay, and the finer-TF publish gate on the same clock.runtime_overrides.timezone/session; defaults (UTC/empty) reduce to bit-identical integer math.Evidence (engine @ 192bf15, codegen @ 726a0f4).
Probe-level: AAPL htf-bucket D 56→86%, 240m 25→61%; EURUSD 240m 13→76% under fx-session grid.
Known follow-ups (next round). EURUSD D/W read-back still diverges: TV exposes the completing HTF candle on the session-final bar with stale-by-two history reads (
close[1]inside security); engine history-offset semantics run one bucket fresh — measured in ledger findings. Wrap-safe overnight eager-completion variant was tested and rejected at population level (net-negative), reverted.Symbol-gate verdict: PASS (four-lane metric gate: tier + entry/exit/PnL p90 regression checks). PR-gate verdict: PASS.