Skip to content

Discover the other symbols' feeds a script requests before it runs - #164

Merged
luisleo526 merged 2 commits into
mainfrom
cg/requests-discovery
Oct 4, 2026
Merged

luisleo526 merged 2 commits into
mainfrom
cg/requests-discovery

Conversation

@luisleo526

Copy link
Copy Markdown
Contributor

What

transpile_full(...)["requests"] lists, before a run, every request site that reads another symbol's feed, so a host can fetch the bars the run needs (one feed per symbol string and timeframe, which the engine matches byte for byte). gate/glue.py's transpile_json success envelope carries the same list, and the input manifest marks input.symbol entries with "kind": "symbol".

{"line": 7, "fn": "request.security",
 "symbol":    {"kind": "input", "title": "Other symbol", "default": "BINANCE:ETHUSDT"},
 "timeframe": {"kind": "chart"},
 "lookahead": false, "gaps": false, "ignore_invalid_symbol": false}

API

Field Value
line, fn The request's 1-based line and Pine function ("request.security"). A helper's request is listed once per symbol and timeframe its call paths pass.
symbol literal (value: the exact key, exchange prefix and .P kept) · input (title: the override key, default) · computed (expr as written, value at the inputs' defaults when literals and inputs compute it, inputs: the override keys it reads) · unresolvable (expr: a deferred refusal; the run stops where its value is read)
timeframe literal (value in the engine's feed spelling: "240", "1D", bare "D" as "1D") · chart (timeframe.period, timeframe.main_period, "") · input (title, default) · computed (expr, value, inputs; a switch of one-expression arms included; an empty value is the chart's)
lookahead, gaps barmerge.lookahead_on, barmerge.gaps_on
ignore_invalid_symbol false when omitted, else its value at the inputs' defaults (null when not computable)
column Only for a request.footprint payload: the feed column it reads ("fp_delta_100_70")

Not listed: a request lowered to na (its value reaches only plots, alerts, tables or logs), a request in a helper nothing calls, and requests that read no other symbol's bars (the chart's own symbol, request.security_lower_tf of another symbol, recorded request.earnings / dividends / splits / financial series). Full text: docs/PUBLIC_CONTRACT.md, "Request discovery (unreleased)"; README "List the other symbols a script requests"; CHANGELOG "Unreleased".

How

pineforge_codegen/request_discovery.py runs after generation and reads what the C++ registers: _security_eval_info's sites lowered onto another symbol's feed (foreign), their symbol and timeframe nodes, tf / tf_expr, gaps and lookahead. Names are expanded with _substitute_tf_input_reads, the expansion that renders the registration-time C++, and a value is folded only over ScriptIndex.registration_value's grammar (literals, input defaults, + == != and or not, ?:, ticker.inherit / standard; for a timeframe also a switch, as _security_tf_switch_runtime_expr renders it). The deferred refusals are the support checker's unpinned requests plus those unpin_requests turns back from a feed, which it now marks unpinned on the request it removes from the program. A helper nothing reaches (security_contexts._reached) is left out. SecurityCallInfo gains the call's loc.

Discovery never fails a script that transpiled: an unknown shape is listed as computed with no value, a deferred refusal's arguments are listed as written (never expanded: the C++ never registered them), folds are memoized per name and capped (256 levels, 100,000 nodes, 1,024-character values), and past the transpile's 120-second budget the remaining sites are listed unexpanded.

Compatibility

Purely additive. The emitted C++ is byte-identical: the engine's 325 public corpus sources and this repository's 540 test-fixture .pine files transpile to the same C++ at main (c5d97ee) and at this head, each in a fresh process (0 diffs; the 51 refused at both are refused with the same message), and so do 25 request-heavy snippets (helper contexts, deferred refusals, a symbol that can select the chart, footprint).

Tests

  • tests/test_request_discovery.py: every symbol and timeframe kind (a literal with an exchange prefix and .P; input.symbol and its override key; computed from literals and from an input default; unresolvable; a plots-only site and an unreached helper left out; timeframe.period, "240", "D" → "1D", "1W"; input and computed timeframes), several sites, one symbol at two timeframes, helper contexts, flags, the footprint column, the manifest kind, the glue envelope, and the shared case (a BINANCE:BTCUSDT 4h chart requesting BINANCE:ETHUSDT at 240 and D): requests names exactly the feeds of its --symbol-feeds index.
  • tests/test_e2e_request_discovery.py: against the engine, the run given exactly the feeds requests names runs; without any one of them it stops with "no data is pinned for this request"; an override of the input by its listed title moves its feeds to the overriding symbol.

Review follow-ups are in the second commit: requests in a helper nothing reaches are left out (registered but never read), a switch timeframe gets the value registration renders, timeframe.main_period is the chart's, and discovery degrades instead of failing past the time budget.

Verification

  • Full suite against engine main 7b596622 (its headers, generated version.h and built libpineforge.a): 5037 passed, 33 skipped, 0 failed. The skips are environmental: 28 comparisons with an earlier codegen commit that the test checkout's history lacks, 4 empty parameter sets, 1 unavailable git commit.
  • Corpus sweep: 314 of 314 compile.
  • Pyodide gate: parity over all 277 fixtures (264 succeed, 13 rejected as expected); the npm package packs.
  • Parity gate over the pinned population, engine 7b596622 with this head: PASS, no improvement and no regression (1009 hard probes, 0 regressions; 6980 target probes, score 0; no probe changed).

Targets the next codegen release; the CHANGELOG entry sits under "Unreleased".

🤖 Generated with Claude Code

luisleo526 and others added 2 commits October 4, 2026 12:45
transpile_full() returns `requests`: every request site that reads another
symbol's feed, with its symbol (literal / input / computed / unresolvable)
and timeframe (literal in the engine's feed spelling / chart / input /
computed) as the generated code keys them before the first bar, plus
lookahead, gaps, ignore_invalid_symbol and a footprint's feed column.
A request lowered to na (display sinks only) or in a helper nothing calls
is not listed; a deferred refusal is listed as unresolvable. The glue's
transpile_json envelope carries the same list, and the input manifest marks
input.symbol entries with "kind": "symbol".

Discovery runs after generation and reads the registered sites; the
emitted C++ is unchanged (the 325 public corpus sources and 540 test
fixtures transpile byte for byte as at main).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
A request in a helper nothing reaches is registered but never read, so
discovery lists neither its feed nor its deferred refusal. A timeframe
switch of one-expression arms gets the value registration renders (an
unmatched switch with no default is the chart's), timeframe.main_period
is the chart's, and past the time budget the remaining sites are listed
as written instead of failing a script that transpiled. A deferred
refusal's arguments are never expanded: the C++ never registered them.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@luisleo526
luisleo526 merged commit b3904ca into main Oct 4, 2026
9 checks passed
@luisleo526
luisleo526 deleted the cg/requests-discovery branch October 4, 2026 05:27
luisleo526 added a commit that referenced this pull request Oct 4, 2026
#164 merged with an Unreleased changelog section and "(unreleased)"
wording; it ships in 1.1.0. Its Request discovery section moves into
1.1.0 with the pull request's link, and the lead names #164 and #162.
The compatibility note no longer says the Python and JSON contract keeps
its result keys: it is additive (transpile_full() and the JSON success
envelope gain requests, input.symbol manifest entries gain kind), nothing
is removed or renamed, and #159's changed enum default stays. #164's
own note joins it: its emitted C++ is unchanged over the engine's 325
corpus sources and the 540 test fixtures, each in a fresh process (51
refused with the same message), which this branch re-checked.

README.md and docs/PUBLIC_CONTRACT.md name 1.1.0 where #164 wrote
"(unreleased)" or "Development builds"; the contract's heading is now
"Request discovery", the anchor the changelog links. The contract's
result-key table, success envelope example and manifest paragraph list
requests and kind, which the error envelope does not carry.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
luisleo526 added a commit that referenced this pull request Oct 4, 2026
* docs: changelog and version scoping for 1.1.0

1.1.0 pairs with engine v1.1.0, whose pineforge.h declares the six
checked settings functions (engine #317) that the C++ of codegen 1.1.0
defines (#159). The changelog section covers #159 and #160: the checked
settings functions, the exception handling at the C ABI and the latched
setter failure, the input.enum default that came out as the enum's first
member, the names reserved for the settings code, the pairing and the
migration step. The Python and JSON contract does not change; the
engine's Docker harness adds applied_runtime.syminfo and two failure
messages for syminfo.mincontract (engine #322).

The README, AGENTS.md / CLAUDE.md, CONTRIBUTING.md,
docs/PUBLIC_CONTRACT.md and docs/pine-cap-activation.md name 1.1.0 where
they named 1.0.1 as the current release or pair, and the pairing table
gets a 1.1.0 row. The release scoreboard sentence renders
releases[1.1.0] from the facts file; the active scoreboard moves to
baseline pineforge-parity-baseline-20261003-engine-dbd17b38. The
anomalies sentence no longer starts its line with a facts marker, so
GitHub keeps it out of a raw HTML block.

Measured for 1.1.0: the engine's 325 corpus sources and the 277 gate
fixtures transpile, with the settings code removed, to 1.0.1's C++ byte
for byte (13 refusals, same messages), each in under 0.1 s on CPython
3.14.6 on an Apple M4 Max. The tutorial run of the quick-start C++
against engine e9f78a6f (Linux x86_64, GCC 13) prints 9 trades and
+738.20, and 13 trades with the old sizing declared.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs: correct the 1.1.0 notes against the code

A review of every claim against #159's code and tests and the engine at
main found these:

- transpile_full()'s inputs and transpile_json report the folded enum
  member as the default of an input.enum whose default is a variable,
  where 1.0.1 reported null: the contract keeps its shape, one value
  changes;
- the enum fold needs a variable set to a literal member, and the C++
  beyond the settings code changes for the fold and the renamed names;
- the receipt gives min, max and step as numbers or null;
- #322 leaves a gridless report's bytes as they were apart from
  elapsed_seconds, but across 1.0.1 and 1.1.0 the fingerprint records the
  versions and the generated C++'s hash, and engine #315 and #316 can
  change trades; the second mincontract message is for a library without
  strategy_set_syminfo_metadata; the fingerprint field is
  fingerprint.provenance.runtime.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs: rewrap a changelog line

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs: the harness symbol feeds and #162 in the 1.1.0 notes

The engine's Docker harness gains --symbol-feeds (PINEFORGE_SYMBOL_FEEDS
in the hub image; engine #327, pineforge-release #23), which installs
other symbols' bars for request.security through C ABI calls that exist
since 1.0.0, so the generated C++ does not change. Its report key
applied_runtime.symbol_feeds and its failure output join the report-key
notes. The documentation bullet names #162, merged since.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs: render 1.1.0's grades from its re-mapped baseline

releases[1.1.0] now names pineforge-parity-baseline-20261004-engine-7b596622
(engine 7b596622, the merge of engine #325, with this repository at
c5d97ee): 7,951 excellent / 38 strong of 7,989. The README's markers are
re-rendered from that facts file (pineforge-release #24), and the
changelog's documentation bullet names the baseline in place of
20261003-engine-dbd17b38. The README's release sentence says the release
"grades", as the engine README does, since 1.1.0 is the current release.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs: fold request discovery (#164) into the 1.1.0 notes

#164 merged with an Unreleased changelog section and "(unreleased)"
wording; it ships in 1.1.0. Its Request discovery section moves into
1.1.0 with the pull request's link, and the lead names #164 and #162.
The compatibility note no longer says the Python and JSON contract keeps
its result keys: it is additive (transpile_full() and the JSON success
envelope gain requests, input.symbol manifest entries gain kind), nothing
is removed or renamed, and #159's changed enum default stays. #164's
own note joins it: its emitted C++ is unchanged over the engine's 325
corpus sources and the 540 test fixtures, each in a fresh process (51
refused with the same message), which this branch re-checked.

README.md and docs/PUBLIC_CONTRACT.md name 1.1.0 where #164 wrote
"(unreleased)" or "Development builds"; the contract's heading is now
"Request discovery", the anchor the changelog links. The contract's
result-key table, success envelope example and manifest paragraph list
requests and kind, which the error envelope does not carry.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant