Repository navigation
Commit 7f807b9
Another symbol's bars reach a run as a feed: kernel instrument feeds + Pine foreign request sites (XSYM-D) (#292)
* XSYM-D: another symbol is a feed -- kernel instrument feeds merged by interval, and the Pine adapter's request sites of other symbols
Kernel (ADR-0001 rule 3, opt-in). NativeRunSpec::instrument_feeds holds other
instruments' own bars, keyed by (instrument, timeframe), each bar with its own
close (close_ms) and optional named columns; a NativeSeriesSource::InstrumentFeed
series names one. On each accepted input the series is handed, in feed order,
every bar not yet handed over that has closed by the input's
NativeInterval::last_traded_close_ms (lookahead off) or opened by its open_ms
(lookahead on); gaps is the subscription contract's; bars before the first
input ride on it. No calendar is guessed for the instrument; the chart side
reads the input's calendar interval (resolved from the label on a raw-label
partition, whose inputs carry no duration). With no feed installed every
digest, delivery and run is what it was. Streams refuse a spec that installs
one. Nine named refusals (NativeRunSpecError/Field, with C twins so the
translation stays exhaustive); C has no route (1.0 boundary row).
Adapter. register_security_eval(sec_id, symbol, tf, input_tf, lookahead_on,
gaps_on, ignore_invalid_symbol) registers a site of another symbol; the host
projects its installed feeds as the spec's instrument feeds, declares one
InstrumentFeed series per site after its same-symbol sites (which keep their
route), and runs the payload on every bar handed over in the requested
context (history and TA over the context's bars, bar_index, time_close and
syminfo.* the context's). gaps_on clears at the bar open, before the COOF
checkpoint. An invalid symbol reads na under ignore_invalid_symbol and fails
closed without it; a missing feed fails closed naming symbol and timeframe;
an aggregated chart is refused. Recorded request series: key -> chart-bar
open -> value, na where the tape has no row.
C ABI: strategy_set_symbol_feed / _feed_column / strategy_set_symbol_facts /
strategy_set_recorded_series, each behind its own PINEFORGE_HAS_*_V1 probe,
reached through BacktestEngine virtuals whose kernel base answers false; a
source host refuses stream_begin while any holds data, and
check_c_abi_runtime.py pins that each declaration says so.
Tests on synthetic bars only: test_native_instrument_feed (kernel, the
report's pinned cases), test_native_instrument_feed_twin (kernel/adapter
twin), test_symbol_request_data (C ABI, recorded series, stream refusal).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
* XSYM-D: run_strategy.py installs a probe's pinned request data from PINEFORGE_REQUESTS_ROOT
The environment contract of workflow docs/xsym-requests.md: when
PINEFORGE_REQUESTS_ROOT is set and <root>/<basename of the strategy dir>/
requests.json exists, the harness holds the manifest to
pineforge-probe-requests/v1 (the refusals of the workflow's
validateProbeRequests) and to this probe, verifies every file it names under
the probe's own files/ against its sha256 (a feed against its byte count),
parses the feeds (an empty or NaN volume or extra value is NaN), the
recorded tapes and the symbol facts, and installs them through
strategy_set_symbol_facts / _feed / _feed_column / strategy_set_recorded_series
before the run. It reads nothing else: never another slug's directory, never
the case-wide PINEFORGE_PINE_LIBRARIES. The run provenance records each
feed's (key, sha, bars), each tape's (key, sha, rows) and the manifest's sha,
as it does for the aux and native security feeds. A malformed manifest stops
the run by name before any engine call; a library without the setters and the
docker runner are refused. Unset, or no manifest for the probe: the engine is
handed exactly what it always was.
The frozen verifier names the probe's build directory BUILD / slug
(pineforge-lab 3bac0b7b scripts/verify-engine-local.py:1695) and runs this
script on it (:1593-1596, :1614-1617) through _run_timed (:190-193), whose
subprocess.run passes no env=, so the runner's environment reaches it.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
* XSYM-D docs: "only the run's own symbol is addressable" becomes the instrument-feed contract
native-engine.md: the field list, the Limits paragraph and the stream-limits
bullet state the new contract; a new section "Instrument feeds: another
symbol's bars" gives the struct, the delivery rule, the validation table, the
stream refusal and the Pine adapter's use (the symbol-key overload, the
payload in the requested context, ignore_invalid_symbol, the fail-closed
refusals, recorded request series, the run_strategy.py environment contract);
the 1.0 C boundary table has the "instrument feeds" row its checker pins.
ADR-0001: the rule-2 ruling row "Wave XSYM-D" (mechanism: another venue is
driven by changing data only; knob: none -- lookahead and gaps are already
the subscription's delivery rules), calibrated by XSYM-DESIGN's 203/203 merge
pins and this lane's two witness reproductions; the HTF paragraph, the run-spec
paragraph and the adapter's section name the new surface. mtf.md, coverage.md
and README: the other-symbol surface, the recorded fundamentals store, the
four C setters; the runtime export counts move 58 -> 62 (66 -> 70 declared,
109 -> 113 in total).
Sentences this commit removes, each superseded by the contract it states:
- ADR-0001 (5691220, lane L14-C): "request.security-style series are
reachable from a bare host, as declared series of the run's own symbol" --
another symbol's bars are reachable now too, and the paragraph says so.
- coverage.md (cc4a56e): "Runtime owns the security state machine, ratio /
calendar aggregation, lookahead / gaps semantics, lower-TF emulation, and
per-security diagnostics", "The runtime owns same-symbol security
computation" and "Why not done yet: Most strategy logic in our test corpus
relies on the chart symbol's bars + indicators" -- the request.security row
and its section state the other-symbol surface instead.
- coverage.md (9507d8f): "Would need a parallel data-ingestion path so the
user can supply a CSV / Parquet of fundamentals", "Future story:
Fundamentals and economic indicators would land as a generic auxiliary
timeline feature" and "Fundamentals- and macro-driven strategies are a
meaningful but smaller user segment" -- the recorded request series store
and the pinned requests manifest are that ingestion path.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
* XSYM-D: doc anchors re-anchored on 1e360bd; ctest floors for the lane's four rows
The rebase onto 1e360bd (INT27 with its round-2 fix) shifted cited lines:
reanchor_doc_citations.py --ref 1e360bd plus check_doc_anchors.py --fix and
four hand fixes bring every anchor back (1725/1725; the diff changes digits
only).
Floors, counted with ctest -N on the lane's tree under each ci_verify
profile's cache definitions:
kernel 292 registered, 291 run (+2: test_native_instrument_feed,
test_run_strategy_requests; the WebSocket row still skips)
release 720 (+4: the two above, test_native_instrument_feed_twin,
test_symbol_request_data)
debug/sanitizers/native registration 701/701/710 (+4 each)
KERNEL_MIN_TESTS 289 -> 291, RELEASE_MIN_TESTS 716 -> 720,
EXCLUDED_REGISTERED_MIN 697/697/706 -> 701/701/710; docs/ci.md and the
floor citations follow.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
* XSYM-D review fixes: an empty tape or feed is installed (na, not unfed); setters refuse a run in progress; the literal input_tf trap deleted
An independent review of 1e360bd..80236d5a found, and this commit fixes:
- The workflow keeps a request that was na on every chart bar as a tape with
a header only (campaign/tv/xsym/decode.mjs keeps non-na rows), and pins it.
strategy_set_recorded_series(key, ..., 0) erased the key, so the first read
failed the run as "no recorded series is installed". n == 0 now installs a
series without rows: every read is na. strategy_set_symbol_feed(..., 0)
likewise installs a feed without bars, whose sites read na, instead of
leaving the symbol unfed.
- The four setters answer -1 while a run is in progress (a stream's warmup
or realtime leg included), naming it: the source host's mutation guard is
inert, so a mid-stream install changed what the run read and its digest.
- strategy_set_symbol_feed_column judges the column name the kernel's way at
install (a name that is not UTF-8 was accepted, then refused at begin).
- register_security_eval(id, "SYM", "tf", "literal", bool[, bool]) bound the
same-symbol overload (const char* -> bool outranks std::string) and read
the symbol as the timeframe. That spelling is deleted; generated code
passes input_tf_ (a std::string) and is unaffected.
- run_strategy.py refuses by name a manifest the schema cannot walk (JSON
nested past the recursion limit, a mintick past a double, an unreadable
file) and a stamp an int64 cannot hold, which ctypes would wrap silently.
- abi-stability.md records the four BacktestEngine virtuals added inside v19
(as N5 did inside v18); mtf.md and native-engine.md state the empty
installs, the in-run refusal and that the generated clear_security() also
clears a gaps_on payload's own history (a codegen limit).
Tests: test_symbol_request_data 77 -> 119 checks (empty tape, empty feed,
column name, in-run refusal, the deleted spelling as static_asserts);
test_native_instrument_feed 10543 -> 12375 (a raw label outside the session
reads the calendar's zero-traded slot); test_run_strategy_requests 13 -> 15.
Fail-before on 80236d5a's headers and library: the static_asserts fail to
compile; 15 runtime failures; the harness raises OverflowError.
Sentences of 4076d7d this commit rewrites: mtf.md's and native-engine.md's
"gaps_on reads na on a chart bar that received no new bar" now also states
that the generated clear_security() resets the payload's own history, and
mtf.md's "Historical runs only: a stream refuses an installed symbol feed"
now also states the in-run refusal.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
* XSYM-D: the symbol-data group comment is the first probe's own doc (detached-comments census 8 -> 0)
The preflight's detached-comments stage (measure_detached_comments.py
--check-ceiling, ceiling 0) counted the eight-line group comment above the
four PINEFORGE_HAS_…_V1 probes as detached: a second block in the gap above
the first probe's own doc. Its text now opens that doc, so the census of the
kernel compile closure is 0 again; the lines keep their numbers.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>1 parent 1e360bd commit 7f807b9
42 files changed
Lines changed: 5326 additions & 440 deletions
File tree
- docs
- adr
- design
- pages
- include/pineforge
- source
- scripts
- src
- source
- tests
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
152 | 152 | | |
153 | 153 | | |
154 | 154 | | |
155 | | - | |
156 | | - | |
| 155 | + | |
| 156 | + | |
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
38 | | - | |
| 38 | + | |
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| |||
436 | 436 | | |
437 | 437 | | |
438 | 438 | | |
439 | | - | |
440 | | - | |
441 | | - | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
442 | 442 | | |
443 | 443 | | |
444 | 444 | | |
| |||
457 | 457 | | |
458 | 458 | | |
459 | 459 | | |
| 460 | + | |
460 | 461 | | |
461 | 462 | | |
462 | 463 | | |
| |||
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
45 | | - | |
| 45 | + | |
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| |||
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
63 | | - | |
| 63 | + | |
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| |||
124 | 124 | | |
125 | 125 | | |
126 | 126 | | |
127 | | - | |
| 127 | + | |
128 | 128 | | |
129 | 129 | | |
130 | 130 | | |
| |||
551 | 551 | | |
552 | 552 | | |
553 | 553 | | |
554 | | - | |
| 554 | + | |
| 555 | + | |
555 | 556 | | |
556 | 557 | | |
557 | 558 | | |
| |||
599 | 600 | | |
600 | 601 | | |
601 | 602 | | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
602 | 618 | | |
603 | 619 | | |
604 | 620 | | |
| |||
888 | 904 | | |
889 | 905 | | |
890 | 906 | | |
891 | | - | |
| 907 | + | |
| 908 | + | |
892 | 909 | | |
893 | 910 | | |
894 | | - | |
895 | | - | |
| 911 | + | |
| 912 | + | |
896 | 913 | | |
897 | 914 | | |
898 | 915 | | |
| |||
0 commit comments