codegen: request.security helpers may read the history of a parameter bound to a derived expression - #262

…ted (#228)

PineForge has no currency-conversion data, so request.currency_rate is no
longer refused at analysis time. It translates to an expression that
evaluates its arguments, stops the run with pine_runtime_error and has the
float type, so a script whose call sits in a branch that never runs
translates and runs, and an evaluated call stops the run like
runtime.error.

The call binds its from/to/ignore_invalid_currency signature through the
existing builtin argument binder; the shared signature diagnostics name
the actual callee (scripts/gen_diagnostics_catalog.py; the generated catalog
follows in the next commit).

Fixes #228

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

… errors (#228)

The short user_message of PF-E1088 / PF-W1088 now reads "The arguments to
{name} do not match its Pine signature." as scripts/gen_diagnostics_catalog.py
writes it, because request.currency_rate shares the nz / fixnan argument
binder.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…or half)

Red-first tests: the reporter's script must translate, explicit and
inferred color declarations are typed from the Pine type, the requested
value must match a helper without var, a color set on one requested bar
must persist when the next bar does not assign it, na colors must match
chart state, and epoch-width int, varip and collection state stay refused.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…color half)

A helper-local var of Pine type color now keeps its state in the helper's
double series: the packed 32-bit ARGB value is exactly representable, na is
preserved, and reads cast back to the ordinary color integer type. The type is
taken from the declaration (type hint, analyzed type, or color-valued
initializer), never from a name or a literal. The epoch-width int64 refusal
and the collection / UDT / drawing refusals are unchanged; their message for
non-int64 types now also names color (new code PF-E3093; PF-E3015 stays
pinned).

Part of #229

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

… part 1)

Red-first tests: the reporter's one-line `set(State s) => s.value := 1`, the
method form, a compound operator and comma-joined statements must translate
to exactly the C++ of the same body written on an indented line; the value
the script plots is 1 on every bar; a plain expression body stays one
ExprStmt with is_single_expr; a plain-name assignment keeps its parser error.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…n or method body (#230, part 1)

`set(State s) => s.value := 1` stopped at the colon-equals token because the
one-line body of a function or method was read as a single expression. The
body now takes the same expression-or-assignment shape as a switch arm's
line, restricted to field and element targets (and their compound operators,
comma-joined statements included). An assignment body is a real statement, so
it is represented as an ordinary block (is_single_expr false) and translates
exactly like the same body on its own indented line. A plain expression body
keeps its one-ExprStmt form, and a plain-name assignment keeps its parser
error.

Part of #230

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…keyword spelling from a test (#228)

The first lab run (rj-20261026t033717-8ad6d7) failed two tests of this branch:

- tests/test_diagnostic_notes.py requires the PF-W1088 user_message to differ
  from PF-E1088's and to say that a switch arm only warns; the catalog
  generator edit had given both the same sentence. PF-W1088 now keeps the
  switch-arm sentence after the callee-aware first sentence, in the generator
  and in the catalog.
- the all-keyword call spelled `to = 'JPY'`, but `to` is a keyword token of the
  lexer and fails to parse as an argument name (ParseError: Unexpected token
  TO). That is a front-end limit independent of this change; the test now
  spells the optional argument by name after two positional ones.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…F-E3093; AGENTS.md lists colour (#229, var color half)

The boundary test (epoch-width int64, varip, collection stay refused) no longer
carries the colour positive control, which is its own test, and its collection
case asserts the new code PF-E3093 instead of matching the old sentence by
prefix. AGENTS.md's helper `var` state row names colour (packed exact 32-bit
ARGB in the double series, na as NaN, read back as int64_t).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

The stop is the repo's own request_stop(request_stop_marker(node)) shim, the
one request.financial / earnings / dividends / splits use, so a new engine
reports the same coded no-data failure for the currency request (function
request.currency_rate, the spelled call and its line) and an older engine
keeps the original pine_runtime_error with the same English text. Arguments
are still evaluated first and a skipped branch evaluates nothing.

The tests derive the expected stop from a transpiled sibling no-data request
instead of a runtime.error twin. AGENTS.md says request.currency_rate is not a
hard reject: it translates and stops where it is evaluated.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…jected shapes; changelog (#230, part 1)

A history subscript is not an assignable target in Pine, so the one-line
function or method body takes only a field (member access) target; an
assignment operator after any other target stays unconsumed and keeps the old
parser error. Two rows pin it: a plain-name second statement after a field
assignment (`s.value := 1, x := 2`) and a subscript target (`s[1] := 2`), both
rejected with the COLON_EQUALS parser diagnostic. CHANGELOG gets the entry.

Part of #230

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

)

A `var` initializer that constructs a UDT (or array/map/matrix, directly or
through a helper) from a variable declared on an earlier line is emitted in
the first-bar preamble, before the statement that assigns that variable, so
it reads the member's zero default: the field holds 0 where the same script
with the dependency written inline holds the value.

These tests fail on the current head for that reason:
- the issue's script: the emitted assignment order of `base` and `state`
  must equal the order of the same script without `var`;
- the issue's script and its inline-expression control, run on eight fixed
  bars through the engine harness (static and dynamic): the positive flag and
  the observed field equal the control's. No trade-level claim is made: the
  eight flat bars place no trade even in the control;
- five dependent aggregate shapes (UDT, array, matrix, map and UDT built by a
  helper) plus persistence, reassignment and moved-scalar-`var` cases, all
  compared against a control form of the same script, never against captured
  codegen output;
- a dependent `varip` row that pins only that the `varip` emission is not
  touched by this change (known gap, not a claim of correctness).

Part of #226.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…tement order (#226)

An aggregate `var` (UDT, array, map, matrix, directly or through a helper)
whose initializer reads a value that only the statement walk computes (an
earlier ordinary declaration, a reassignment or field write, a collection
method, or a global read by a user helper) used to be initialized in the
first-bar `_var_initialized` preamble, before those statements ran, so it
read the member's zero default. Silent wrong value.

Such declarations now use the existing per-declaration once guard at their
statement position:
- base.py selects them (`statement_ordered_aggregate`) using traversal
  positions and the analyzer's binding scopes, so helper locals and
  parameters cannot capture an unrelated global of the same spelling;
  later aggregate dependencies on a moved `var` follow it;
- emit_top.py skips the selected RHS in the preamble and pushes the typed
  na slot for a history-read UDT;
- visit_stmt.py supplies the authored UDT target type and avoids numeric
  coercion for the routed aggregates.
Independent constructors, aliases, literals, bar-only initializers, `varip`
and callable-local declarations keep their existing routes; non-var
statements keep their order. Existing tests and expected fixtures are not
changed. AGENTS.md and CLAUDE.md state the support contract in one paragraph.

Part of #226.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…est.security evaluation

A lab probe built only from shipped features: calc_on_order_fills, two
same-symbol request.security calls (30m lookahead off, 5m lookahead on) whose
helper keeps a var counter, and a late margin-call liquidation after the
script walk. The requested counters are compared with a pure-Python count of
completed requested buckets on every bar, and with a control run without the
margin call. It asserts that the stressed run really executed the margin fill
and the restore. It fails on the current tree: the counter reads 11 where the
feed gives 13.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…ss a late restore

tests/test_security_coof_restore_members.py transpiles scripts with a
finer lookahead request (numeric and string helper state, TA binding
variants, OHLC and TA histories with the bar-index clock, mutable globals,
expression and session histories) and asserts on the emitted rollback hook:
requested-progression members are copied but not restored, result cells and
chart members are restored, a chart-fallback payload keeps its members in the
restore, and tuple and lower-timeframe result cells still restore. Every
expectation is derived from the emitted code of the same script.

tests/test_e2e_security_coof_restore_history_reads.py runs the chart reading
offset history of a requested series across the late margin restore and
compares with a feed-derived oracle.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…-evaluation state

Under calc_on_order_fills the engine commits the script-state checkpoint
before it feeds the deferred requested evaluations of a chart bar. A later
fill recalculation (for example a late margin call) restored that older
checkpoint, so the generated state of the request.security context (helper
series and their var seeds, requested TA objects, OHLC and expression
histories, mutable copies, bar-index clocks) lost the deferred writes while
the engine's requested aggregators kept advancing: a silent wrong value.

The generated restore_script_state() now skips exactly those members, taken
from the same metadata that declares them, and still restores chart state and
the merged request results. Snapshot and commit are unchanged. A payload that
keeps user calls on the chart's bar (the chart-fallback warning) keeps its old
rollback, and so do the shared helper maps of a script that has one. No engine
or ABI change. Over the 555 corpus and fixture scripts the emitted C++ differs
from the parent revision in 72 of them, and only inside that one function.

Pins that hash the whole emission now hash it with the rollback hook body
removed, so they keep proving that everything else is unchanged:
- map source-order script 1: old (full emission) 8c6ee0d0..., new (hook body
  removed) 25368130..., reason: the hook body differs by design; the stripped
  emission of the parent revision gives the same 25368130...
- map source-order script 2: old acd85265..., new 074bda25..., same reason and
  the same equality with the parent revision.
The whole-C++ identity test against the parent compares with the hook body
removed on both sides (shared helper without_restore_script_state).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Brings in the 1.8.0 release notes. The release note for this fix is a
separate 1.8.1 (planned) section above the planned-1.8.0 section.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

Resolve the CHANGELOG conflict with the planned 1.8.0 section: the entry for
the statement-ordered persistent initializer sits in a new "1.9.0 — planned"
section above it, and the 1.8.0 section is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…ted-evaluation state

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

The CHANGELOG entry of this change is carried by the combined 1.8.0 entry
written in the last commit of this branch.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

The CHANGELOG entry of this change is carried by the combined 1.8.0 entry
written in the last commit of this branch.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

The CHANGELOG entry of this change is carried by the combined 1.8.0 entry
written in the last commit of this branch.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…r method

The CHANGELOG entry of this change is carried by the combined 1.8.0 entry
written in the last commit of this branch.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

The planned-1.8.0 section now carries the five entries (late-restore rollback,
statement-order initialization of persistent declarations, request.currency_rate
evaluated only where it runs, helper-local var color, one-line field
assignment), updates its summary and its emitted-C++ and diagnostics notes, and
the separate planned-patch section added by one of the merged branches is
removed. The compatibility bullet scopes the restore-function figure to the
late-restore change and names both reworded diagnostics (PF-E1088, PF-W1088).
A test comment no longer names an internal system.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

… helper-parameter history)

Red-first specification for the reporter's script and its derived-argument
variants (arithmetic, nested helper, two call sites, one site read twice,
other constant offsets, a same-slot revision), with the expected vectors
computed in the test from the inputs. The helper-free global/TA history path
and the value-only helper are included as controls that already pass.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…n named, lookahead-on reporter (#229, helper-parameter history)

The same-slot revision row now asserts one completed evaluator row per
requested slot on every feed and strictly more evaluator calls at the last
slot on finer feeds, instead of a typed call count. The push/update site check
is named a shape pin of design A. A lookahead-on row for the reporter's script
pairs the lookahead-off default.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

… helper-parameter history)

Adds PF-E3094 through PF-E3101 to the catalog and its pins: the located
refusals of a request.security helper whose parameter history cannot be
sampled (derived argument with a non-constant or negative offset, capacity not
representable, conditional call, dependency-aware helper, composition with
helper-result history, mutable global that cannot be replayed, non-numeric
scalar, parameter type or width without a lossless requested argument
history). The messages are unchanged by this commit's code numbers; the
emitting code follows in a later commit of this branch.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…pe (#229, helper-parameter history)

Pins every refusal of a request.security helper whose parameter history
cannot be sampled: the code, the message and the line/column of the call, and
the shapes that stay admitted or keep their previous lowering (builtin
history, method envelopes with the chart fallback, a helper call through a
global value).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…ecurity (#229, helper-parameter history)

A request.security helper that reads the history of a parameter bound to a
derived expression (for example f(close - open) with x[1] in the body) now
samples the argument on the requested context's own bars through a per-call-site
ring of the argument values, instead of reading the chart's bars. Admission is
by the position of the call; the shapes that cannot be sampled are refused with
located errors (PF-E3094 through PF-E3101). The ring is requested-evaluation
progression state, so the restore after a calc_on_order_fills fill leaves it
alone like the rest of that state. The history of a fully overwritten global
source uses the last unconditional write before the request statement.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…ips in 1.8.0

Adds the entry (sampling on the requested context's own bars through one ring
per call site; the located refusals PF-E3094 through PF-E3101), extends the
compatibility bullet with the scripts and the nine added diagnostic codes.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>

…e engine

The witness turned a missing native engine into a failure, so the pytest jobs
of this pull request, which run without the engine, failed. It now skips like
the other late-restore witnesses; the full suite with the paired engine runs it
for real.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>