Repository navigation
Fail the run when a timezone lookup cannot be carried out - #840
Conversation
A strategy whose chart timeframe is N days (2D, 3D, 5D ...) built from daily bars ran on the days-since-1970 grid, and the chart's bar count, last_bar_index, barstate.islast and open-trade row came from the wrong bars. TradingView builds an N-day chart from N consecutive trading dates counted from the first trading date of each calendar year (the last bucket of a year is cut at its end), and its last_bar_index, barstate.islast and open-trade row follow that chart's own last bar. The engine already had that partition for request.security; this change installs it for the chart itself. The scheduler creates the existing partition source for an aggregated chart with a day count of two or more and hands it to the execution consumer, whose script bucket cursor, interval lookup, chart close and chart-time offsets read it. Only this chart path drops a leading bucket that opens before the input range and seals the final historical bucket. The calendar gains an interval overload that takes the partition's adjacent opens. The epoch-key paths, request.security, the D, W and M charts, intraday charts and runs whose input and chart timeframes are equal do not change, and a run without a partition adds nothing to the state hash. A day count whose seconds do not fit the legacy int conversion is calendar-only. Limits: a live stream keeps the previous chart path, and a selected window on a generated N-day chart partition is refused with the existing coded stop window_mode_unsupported (capability selected_window_chart_timeframe), because the window planner reconstructs the nominal calendar. Time offsets, exchange calendars and intraday input are not measured against TradingView. tests/test_n_day_chart_timeframe.cpp holds the rows on synthetic daily bars: the 3D grid, the last bar index, barstate.islast, the open row at the end, the 2D and 5D grids, the stream and refusal rows, the time offsets and the controls that must not move. It is registered through tests/cr150.d/n-day-chart-timeframe.cmake. scripts/check_native_cpp_versions.py lists the new reads of the retained input bars in the scheduler. TradingView tapes (sha256): 3D chart 1b2ef41a6f1ba4096e6659e12d5ef3f03c8bf2a51e4bb0d8331d814d9aade262, 2D chart ea679d91cbc0cd078c80604e2c73b3c9bb3104ff8d1989695da6abecb8b9e174, 5D chart 5342be10bdda8185b12b2c6a95b9b26101334cd1f58663ceb1f28c42197cb5dc, last_bar_index 05617c83d33cbd8be7a0d64356dc8f40e5b8d2ab20baa1b74b08b8aa66a07d25, barstate.islast 4ae8e0a932b7da6ffb4e0bcd5de9ee4d514cd6b78dbbb3fa4207aba1a2c6420d. Refs #811. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The N-day chart commit moved code in src/native_execution_consumer.cpp, src/native_execution_consumer.hpp, src/source/pine_strategy_host.cpp, src/source/pine_scheduler_native.cpp and src/timeframe.cpp, so the file:line citations of the published pages that point into them went stale, and scripts/check_doc_anchors.py reported 212 bad anchors. This commit changes digits only: the line numbers inside file:line citations of nine pages (145 changed lines, 233 changed numbers) and nothing else. No sentence, word, content pin or table is edited. The numbers were moved with the repository's own tools: scripts/check_doc_anchors.py --fix (the cited symbol sits on exactly one line) and scripts/reanchor_doc_citations.py --ref 360c4ab (a line map from the commit before the N-day chart change, where the pages were correct). The last six citations, which claim no symbol, were carried across the same line map by hand: the cited code line is the same text at its new number. scripts/check_doc_anchors.py reports no bad anchor on the result, and scripts/check_doc_reverts.py reports nothing, since a re-anchor is not an edit. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Main brings the memory-failure, timezone and setter-error changes. The conflicts were the Unreleased list in CHANGELOG.md, where both sets of bullets are kept unchanged, and the line citations of the published pages, which take main's lines here; the citations are re-anchored in the next commit by the anchors gate's own fix. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Merging main moved code in src/native_execution_consumer.cpp, src/source/pine_strategy_host.cpp, src/source/pine_scheduler_native.cpp and the other files the N-day chart change touches, so the file:line citations of the published pages that point into them went stale, and scripts/check_doc_anchors.py reported 208 bad anchors on the merge. This commit changes digits only: the line numbers inside file:line citations of nine pages (140 changed lines) and nothing else. No sentence, word, content pin or table is edited. The numbers were moved with the repository's own tools: scripts/check_doc_anchors.py --fix (the cited symbol sits on exactly one line) and scripts/reanchor_doc_citations.py --ref e163980 (a line map from main, where the pages are correct). The last six citations, which claim no symbol, were carried across by hand: the cited code line is the same text at its new number (one of them, the cursor constructor, gained a parameter on its line). Co-Authored-By: Claude Opus 5.5 <[email protected]>
The check that accepts a timezone name written in a script looks the name up in the timezone database with realpath, open and read, and it treated every failure of those calls as "no such zone". When the process had no file descriptor or memory left, or a read failed, a valid name such as America/New_York was refused with pine_timezone_error (class strategy, not retryable) and TradingView's text for an incorrect timezone; a database directory that could not be resolved refused every zone name the same way. The lookup now tells an absent name from a lookup that could not be carried out. ENOENT, ENOTDIR, ELOOP, ENAMETOOLONG and EISDIR still mean that the name denotes no zone file and keep pine_timezone_error. ENOMEM raises out_of_memory. Every other error raises engine_unclassified_error with the errno kept and the failed call named in the message. When the database directory itself cannot be resolved, a missing directory included, the run fails the same way and the message names the directory. In this check an interrupted call is tried up to three times, and a header read that returns fewer than four bytes is continued. Only the argument check of script timezones raises, through the internal timezone_lookup_accepted; the installed native_calendar::timezone_accepted keeps its false result, so its callers, the selected-run admission among them, refuse as before. A name that is also a complete POSIX definition, such as EST5EDT, stays accepted when its optional file lookup fails. A failed lookup is not remembered as an accepted name. tests/test_timezone_lookup_failure.cpp interposes realpath, open and read to fail each call of the lookup with each class of error, and checks the failure code, the errno and the call named in the message, that the name is looked up again afterwards and a clean run then books the same trades, that one interrupted call does not change the run, the file header (wrong magic, end of file, a read split in two), the database directory, the selected-run admission, and a TZDIR that is not a directory, where UTC offsets and POSIX definitions still run. Documentation sentences this change removes or rewrites: - docs/coverage.md, the pine_str_format_time row: "The message is TradingView's text with the engine's bar prefix, `Error on bar N: `, or `Error before the first bar: ` when no bar is being executed, and the arguments carry `bar_index`." It now begins "The error message for a refused name is", after a new sentence on the lookup failures. The allocation-bound manifest lists one new group, the message string of the lookup refusal in the native calendar (ceiling 1418 -> 1419, pinned count 1356 + 63 + 184), and its site inventory is re-emitted. Co-Authored-By: Claude Opus 5.5 <[email protected]>
A failed void C setter records its failure in the handle's run consumer. If a handle had no consumer, recording would first allocate one, and a failure of that allocation would leave only the status, without the code or the text. That state cannot be reached: the engine's only constructor binds the consumer, and the native host can be neither copied nor moved. These tests pin that, and show that the record survives a second allocation failure, without a change to the engine. - Every host type of the test starts with its consumer: storing a selected-window descriptor allocates nothing and does not return the allocation refusal -4. - When a setter's own string and then the rendering of the record's bar prefix are both refused, the setter's out_of_memory record is read without any allocation and its text falls back to "setter failed"; the next run start fails once with that record and publishes no report, and the start after it produces the report of a clean run. A setter refusal that carries its own code keeps that code when the rendering fails. - A sweep refuses two, and then three, consecutive allocations starting at every allocation index of a configured run (the setters, preparation, the bar callbacks and the report). Each injected run ends in a typed setup refusal before the first bar, in out_of_memory after it, or in a report byte-identical to the clean run, and the sweep prints how many refusals each run received: a pair at 256 indexes (Debug 258), a triple at 37 (Debug 39); the other 234 end at the first refusal. tests/fail_alloc_seam.hpp gains arm_span(first, count), which refuses count consecutive allocation attempts, and refusals(), the number of refusals delivered; arm(nth) is arm_span(nth, 1). Co-Authored-By: Claude Opus 5.5 <[email protected]>
Main brings the description of the current CI job graph in AGENTS.md and CLAUDE.md. Neither file is touched by this branch, so the merge has no conflicts and changes nothing else. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Main gained the CI job graph documentation (AGENTS.md and CLAUDE.md); these changes touch neither file. Co-Authored-By: Claude Opus 5.5 <[email protected]>
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; formal verification proved 0 of 6 changed function(s) (6 not verified).
Formal verification. PR-changed functions: 0/6 verified (0 proven, 0 may-equivalent, 0 distinguished) · 6 not verified (6 unsupported).
Not verified on this run: timezone\_classify (unsupported), tz\_file\_actual\_path (unsupported), tz\_file\_is\_tzif (unsupported), tzdir\_canonical (unsupported), tzdir\_realpath\_dir (unsupported), resolve\_script\_timezone (unsupported).
Graphify review — findings
Fails the run with an engine error when a timezone database lookup hits fd or memory exhaustion, an I/O error, or a missing database, instead of reporting a wrong timezone name. The native calendar's path-resolution helpers (tzdir_realpath_dir, tzdir_canonical, tz_file_actual_path) now take an optional TzLookupFailure out-parameter, and tz_raise_lookup_failure turns a recorded failure into a short typed message naming the directory or file. New tests interpose open, realpath and operator new to exercise each failure mode. The CR150 source-bounds gate registers the new message-building sites as one more unreviewed group, raising the ceiling from 1418 to 1419. A CHANGELOG entry documents the behaviour change.
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 933 functions depend on the 593 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
tf_change()— 28 callers, 5 callees - worse:
crosses_boundary()— 7 callers, 8 callees - worse:
build_native_day_partition()— 9 callers, 5 callees - worse:
configure()— 7 callers, 6 callees - worse:
interval_containing()— 10 callers, 4 callees - worse:
timezone_classify()— 4 callers, 9 callees - worse:
session_day_at()— 8 callers, 4 callees - worse:
timeframe_time_close()— 2 callers, 12 callees - …and 11 more — each is listed as a finding
Verification — 933 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 769 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
1424 of 1424 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety (
noncode-change,uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.
benchmarks/tests/test_aggregate.py— full-run-safetybenchmarks/tests/test_check_provenance.py— full-run-safetybenchmarks/tests/test_compare.py— full-run-safetybenchmarks/tests/test_public_names.py— full-run-safetybenchmarks/tests/test_run_all.py— full-run-safetybenchmarks/tests/test_run_strategy_feeds.py— full-run-safetydocker/pineforge_sink_finalize_test.py— full-run-safetydocker/pineforge_sink_replay_rq_test.py— full-run-safetydocker/pineforge_sink_replay_test.py— full-run-safetydocker/pineforge_sink_to_json_test.py— full-run-safetydocker/quota_columns_test.py— full-run-safetydocker/quota_convert_test.py— full-run-safetydocker/quota_finalize_test.py— full-run-safetydocker/replay_rq_engine_test.py— full-run-safetydocker/run_json_artifact_window_test.py— full-run-safetydocker/run_json_bin_test.py— full-run-safetydocker/run_json_bounded_transport_test.py— full-run-safetydocker/run_json_bulk_equity_fuzz_test.py— full-run-safetydocker/run_json_bulk_equity_test.py— full-run-safetydocker/run_json_codes_test.py— full-run-safetydocker/run_json_converter_metadata_test.py— full-run-safetydocker/run_json_converter_session_test.py— full-run-safetydocker/run_json_diagnostics_test.py— full-run-safetydocker/run_json_equity_scan_test.py— full-run-safetydocker/run_json_fused_test.py— full-run-safetydocker/run_json_harnesses_test.py— full-run-safetydocker/run_json_metrics_only_test.py— full-run-safetydocker/run_json_misc2_test.py— full-run-safetydocker/run_json_misc_test.py— full-run-safetydocker/run_json_pfbar_test.py— full-run-safetydocker/run_json_provenance_test.py— full-run-safetydocker/run_json_quota_supervisor_test.py— full-run-safetydocker/run_json_replay_test.py— full-run-safetydocker/run_json_selected_sink_test.py— full-run-safetydocker/run_json_sink_lexical_test.py— full-run-safetydocker/run_json_sink_summary_test.py— full-run-safetydocker/run_json_sink_test.py— full-run-safetydocker/run_json_sink_vectors_test.py— full-run-safetydocker/run_json_spool_work_test.py— full-run-safetydocker/run_json_stringio_test.py— full-run-safetydocker/run_json_symbol_feeds_test.py— full-run-safetydocker/run_json_syminfo_test.py— full-run-safetydocker/run_json_trade_scan_test.py— full-run-safetydocker/run_json_window_budget_test.py— full-run-safetydocker/run_json_wrapper_work_test.py— full-run-safetyinclude/pineforge/native_run_spec.hpp— full-run-safetyscripts/derive_corpus_feeds_self_test.py— full-run-safetyscripts/fingerprint_self_test.py— full-run-safetyscripts/run_corpus_self_test.py— full-run-safetyscripts/test_abi_receipt_skips.py— full-run-safety- … and 1374 more
non-code file(s) changed (
CHANGELOG.md,ci/cr150-source-bounds.yaml,ci/cr150-source-sites.json,docs/coverage.md,docs/pages/run-failure-codes.md…) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
CHANGELOG.md,docs/coverage.md,docs/pages/run-failure-codes.md,src/timezone.hpp,tests/fail_alloc_seam.hpp) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
Could not verify: Could not verify timezone\_classify.
The verifier did not have enough to check timezone\_classify, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported: signature arity changed (1 -> 2 positional params)
Could not verify: Could not verify tz\_file\_actual\_path.
The verifier did not have enough to check tz\_file\_actual\_path, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported: signature arity changed (1 -> 2 positional params)
Could not verify: Could not verify tz\_file\_is\_tzif.
The verifier did not have enough to check tz\_file\_is\_tzif, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported: signature arity changed (1 -> 2 positional params)
Could not verify: Could not verify tzdir\_canonical.
The verifier did not have enough to check tzdir\_canonical, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported: signature arity changed (0 -> 1 positional params)
Could not verify: Could not verify tzdir\_realpath\_dir.
The verifier did not have enough to check tzdir\_realpath\_dir, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported: signature arity changed (2 -> 3 positional params)
Could not verify: Could not verify resolve\_script\_timezone.
The verifier did not have enough to check resolve\_script\_timezone, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: unsupported parameter type(s)
· 9 grounded finding(s) anchored inline below; 10 more finding(s) on lines outside this diff (see the check run).
| // one, observational descriptor and optional acceleration callers retain one | ||
| // attempt per interrupted call and nullopt for failures. Both modes accumulate | ||
| // partial headers. Argument validation requests typed failures. | ||
| std::optional<std::string> tz_file_actual_path(std::string_view name, |
There was a problem hiding this comment.
tz_file_actual_path()
high coupling complexity (Ca·Ce = 12).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| } | ||
|
|
||
| TzClass timezone_classify(std::string_view tz) { | ||
| TzClass timezone_classify(std::string_view tz, bool report_failure = false) { |
There was a problem hiding this comment.
timezone_classify()
fans out to 9 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| bool escaped = false, report = false; | ||
| }; | ||
|
|
||
| Outcome run(const char* zone = valid_zone) { |
There was a problem hiding this comment.
run()
high coupling complexity (Ca·Ce = 15).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| } | ||
|
|
||
| #if defined(__linux__) | ||
| void errno_rows(const Outcome& clean) { |
There was a problem hiding this comment.
errno_rows()
fans out to 8 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| #endif | ||
|
|
||
| #if defined(__GLIBC__) | ||
| void missing_directory_rows(const Outcome& clean) { |
There was a problem hiding this comment.
missing_directory_rows()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| // Other platforms compile the clean-run and invalid-name controls below. | ||
| #endif | ||
|
|
||
| int main() { |
There was a problem hiding this comment.
main()
fans out to 17 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| CHECK(read_attempts == 0 && !read_fired); | ||
| } | ||
|
|
||
| void setter_failure_survives_rendering_failure() { |
There was a problem hiding this comment.
setter_failure_survives_rendering_failure()
fans out to 14 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| CHECK(report_bytes(following.value) == report_bytes(expected.value)); | ||
| } | ||
|
|
||
| void rendering_failure_keeps_setter_code() { |
There was a problem hiding this comment.
rendering_failure_keeps_setter_code()
fans out to 9 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| std::size_t configure_allocation = 0; | ||
| }; | ||
|
|
||
| SpanOutcome complete_span_run(std::size_t first, std::size_t span) { |
There was a problem hiding this comment.
complete_span_run()
fans out to 8 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
…follow-ups The release lands #829 first; this branch carries that change's head, so that taking main after its squash merge changes nothing in this tree. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Main now carries #829's squash merge, whose tree equals the #829 head this branch already merged, so this merge changes nothing in the tree. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…tep fix The release lands #829 and #840 first; this branch carries their heads, so that taking main after their squash merges changes nothing in this tree. The cited line numbers that both sides moved are combined (digits only). Co-Authored-By: Claude Opus 5.5 <[email protected]>
Engine main took the change this branch was stacked on (#840) as one squash commit whose tree equals the #840 head this branch already carries, so this merge changes nothing in the tree. Co-Authored-By: Claude Opus 5.5 <[email protected]>
What was wrong
The check that accepts a timezone name written in a script looks the name up in the timezone database with
realpath,openandread, and it treated every failure of those calls as "no such zone". When the process had no file descriptor or memory left, or a read failed, a valid name such asAmerica/New_Yorkwas refused withpine_timezone_error(class strategy, not retryable) and TradingView's text for an incorrect timezone; a database directory that could not be resolved refused every zone name the same way.The void C setters record their failure in the handle's run consumer. That this record survives a second allocation failure (the setter's own allocation and then the rendering of the record's bar prefix) was not tested.
What changes
ENOENT,ENOTDIR,ELOOP,ENAMETOOLONGandEISDIRstill mean that the name denotes no zone file and keeppine_timezone_error.ENOMEMraisesout_of_memory. Every other error raisesengine_unclassified_errorwith the errno kept and the failed call named in the message; a database directory that cannot be resolved, a missing one included, fails the run the same way and its message names the directory. In this check an interrupted call is tried up to three times, and a header read that returns fewer than four bytes is continued.native_calendar::timezone_acceptedkeeps itsfalseresult, so its callers, the selected-run admission among them, refuse as before. A name that is also a complete POSIX definition, such asEST5EDT, stays accepted when its optional file lookup fails. A failed lookup is not remembered as an accepted name.Scope in plain words
No new failure code, function, field or macro: the lookup failures use the existing
out_of_memoryandengine_unclassified_error. Runs whose timezone lookups succeed, and runs that use no named zone, are unchanged.Tests
tests/test_timezone_lookup_failure.cpp(new) interposesrealpath,openandreadto fail each call of the lookup with each class of error, and checks the failure code, the errno and the call named in the message, that the name is looked up again afterwards and a clean run then books the same trades, that one interrupted call does not change the run, the file header (wrong magic, end of file, a read split in two), the database directory, the selected-run admission, and aTZDIRthat is not a directory, where UTC offsets and POSIX definitions still run.tests/test_void_setter_failure.cppgains rows for every host type of the file (each starts with its consumer), for a setter whose own string and then the rendering of the record's bar prefix are both refused (theout_of_memoryrecord is read without any allocation; the next run start fails once with it and publishes no report; the start after it produces the report of a clean run), and a sweep that refuses two, and then three, consecutive allocations starting at every allocation index of a configured run: each injected run ends in a typed setup refusal before the first bar, inout_of_memoryafter it, or in a report byte-identical to the clean run. The sweep prints how many refusals each run received: a pair at 256 indexes (Debug 258), a triple at 37 (Debug 39); the other 234 end at the first refusal.tests/fail_alloc_seam.hppgainsarm_span(first, count)andrefusals().Release
A patch-level change (no catalog change). One
CHANGELOG.mdbullet in the unreleased section.Squash commit message
🤖 Generated with Claude Code