Skip to content

run_json: --metrics-only, a certified module takes the no-curve metrics route - #630

Closed
luisleo526 wants to merge 12 commits into
mainfrom
core/metrics-only-chain
Closed

luisleo526 wants to merge 12 commits into
mainfrom
core/metrics-only-chain

Conversation

@luisleo526

@luisleo526 luisleo526 commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

What changes

run_json gains --metrics-only[=auto|legacy] (a bare option means auto). The document it prints is the ordinary document without trades, equity_curve, elapsed_seconds and the two scratch-arena byte counters; every other key keeps its name, value and relative order. Without the option nothing changes.

With auto, a native module that certifies itself takes the engine's ordered-replay metrics route, so a run no longer keeps the full per-bar equity curve: Sharpe, Sortino, the monthly ratios, the equity summary and the trade statistics are computed as the run goes. The document is byte for byte the one legacy prints. Commits, in order:

  1. Monthly ratios on the no-curve route: they replay from bounded month-end state (at most 4096 closed months), bit-identical to the curve route, with rows against the curve route.
  2. Native module wrapper: a host certifies itself with an exact-type specialization of its plain-run declaration; the run wrapper calls admission and preparation before the run only when the preference is on and the host certified; one new optional C export, strategy_metrics_only_capability_v1, reports it (0 certified, -1 not). The ABI inventory and page list it. A host that does not specialize is never treated as plain.
  3. run_json: option parsing and refusals (an explicit report sink, a phase stream, selected-window options, outputs, a benchmark run, symbol feeds), the capability probe (only status 0 counts; a missing export, any other status or a failing call means no), mirrors of the receipt structs (size-checked at import), whole-run recovery (one repeat from the frozen inputs when a receipt says the curve is needed), and the document shape.
  4. The receipt carries the engine's own final equity mark (actual_last_equity and a validity word): the receipt grows from 232 to 248 bytes, appended; plan, extent and stop structs, the plan comparison and the admitted-view hash are untouched. run_json accepts a curve-free result with a finite annualized return only with a valid finite final mark and the module's exported initial capital, and builds the annualization diagnostic with the existing helper from a two-row stand-in curve (one copy of the arithmetic).
  5. Real-process rows: two native test modules (one certified, one control) driven by the real docker/run_json.py in the default and jailed profiles over a six-year tape: auto and legacy stdout are byte-identical, the ordinary document projects onto the scalar document, the destruction-time receipt shows the route actually taken, and a Release-only memory row (1,000,000 and 3,000,000 bars, three fresh processes per cell) requires a saving of at least 12 bytes per bar for the certified module and under 4 for the control.
  6. The range-provenance reader in the engine scripts reads rangeProvenance first; the old member name is accepted until the next minor.
  7. Documentation citations and source-site gate data re-pointed for the moved lines (three new manifest entries, all unreviewed), and the changelog entry.

Verification

Measured on CLOUD: the real-process rows failed before the receipt field existed (two destroyed handles, 0.06 bytes per bar saved) and pass after; mutants that make the capability probe always false or always true turn the certified and the control rows red respectively; the range-provenance mutants (fallback deleted, new name ignored) turn their rows red. Full legs on the rebased code (release 1406 tests, debug 1311, sanitizers, kernel, parity 312 of 312 probes and the subset), and preflight 59 of 59 stages.

🤖 Generated with Claude Code

luisleo526 and others added 10 commits October 28, 2026 23:57
… no-curve run covers multi-month histories

The ordered return replay removed the per-bar equity curve only for a run whose need mask
asked the bar Sharpe and Sortino alone and whose history stayed inside one calendar month:
with the curve dropped, a second month sent the run back through the curve (whole-report
recovery). The monthly Sharpe and Sortino now come from bounded state: the equity at each
closed month's end (at most 4,096 closed months, plus the current and held month's scalars;
about 32 KiB steady, 48 KiB while the array grows), folded with the legacy operation order
over the per-month returns, so the values are the legacy values bit for bit.

- The largest no-curve mask is now every need but Returns and Curve (0x37). The default
  mask still asks Returns and Curve and keeps the curve; no default and no public struct
  size changes.
- The two-month recovery trigger fires only when the monthly replay did not complete: a
  larger month population than the bound, a reversed month key, an allocation failure, an
  interrupted run, or a planned endpoint outside the range the month key computes in without
  overflow (those keep the curve).
- The month-key helper in the engine header guards the speculative month-key lookups the
  recorders make before the monthly fold can refuse a timestamp; no member layout or virtual
  slot changes. The month key's own integer arithmetic is not changed here.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
New test_metrics_monthly_replay (78 rows) compares the monthly-replay route with the legacy
route computed from the real actual curve in the same binary (streaming switch off), bit for
bit and for every field of the equity summary, with an independent reference where the
neighbouring rows have one: generated series of 1, 2, 3, 12 and 60 months, month boundaries
at the exact last and first bar, a one-bar month, a chart timezone whose local month differs
from the UTC month, flat months, gaps of empty months, negative, zero and heavy-tail months,
the Sharpe-only, Sortino-only and both masks and the largest no-curve mask, stopped runs,
interrupted execution keeping the whole-report recovery, a month population above the bound,
and the default mask keeping every requested curve point. Registration through its own
tests/cr150.d lane file.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…one capability export

A loaded native module can now opt in to the no-curve metrics route. A host certifies itself by
specialising `native_module::MetricsPlainHostV1<Host>` (exact type: a derived host does not
inherit it) with its truthful `MetricsPlainDeclarationV1`; the generic macro declares no host plain
on its own. When the engine's metrics preference is On and the host is certified, the run wrapper
derives the planned extent and the plain-run inputs from the invocation (count, first and last bar
time, the staged native spec, the timeframe arguments, report policy, subscriptions and feeds),
binds the input, spec and view with fixed-width digests, and calls the engine's plain-run
admission and preparation immediately before the run. Anything else (preference Off, an
uncertified host, a throwing or unknown declaration, a non-plain answer) leaves the engine on the
actual-curve route exactly as before, and the block never throws.

New optional C export `strategy_metrics_only_capability_v1()`, generated for every module: it
returns Ok (0) for a certified host and NotImplemented (-1) otherwise, so a loader probes by
calling it. No existing export, struct size, default or output changes. The runtime export
inventory check pins the generated module surface separately from the static runtime exports, and
the stability page documents the probe, the specialisation shape and the recovery contract.

Sentence replaced in docs/pages/abi-stability.md, named for the revert check: "A built strategy
`.so` exposes 110 compiled-strategy `PF_API` declarations (94 runtime implementations plus sixteen
generated exports) plus 43 native-" (the stability page now states the aggregated inventory counts
and lists the optional module export separately; introduced by 95f4fd1).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ts, on and off)

New test_native_module_metrics_only builds two small concrete hosts through the real module macro,
one certified and one not, over a 48-bar, twelve-month feed with long and short trades, fees and a
terminal open position. Rows: the capability export returns Ok and NotImplemented for the two
hosts and a subclass cannot inherit certification; a certified host with the preference On reaches
Streaming through both generated run entries with every scalar metric word-identical to an
independent actual-curve control (and no curve retained); an uncertified host, the Off preference
and the default all-needs mask keep the actual curve and the same words; a selected window and a
synthesised intrabar path are not plain and keep the curve; a throwing or unknown-version
declaration cannot escape or admit; the input, spec and view identities are stable for frozen
input and change with an interior volume word or the fee setting; a stopped run keeps the
whole-run recovery receipt. Registered through its own tests/cr150.d file.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ute, with whole-run recovery

New option `--metrics-only[=auto|legacy]` (a bare option means auto). The document it prints is
the ordinary document without `trades`, `equity_curve`, `elapsed_seconds` and the two scratch
arena byte counters; every other key keeps its name, value and relative order. Without the option
nothing changes: requests, errors, checks and defaults are byte for byte what they were.

`auto` asks the engine for the streaming preference (on, need mask: Sharpe, Sortino, monthly
ratios, equity summary, trade statistics) only when the loaded module certifies itself: the
module's own capability export (`strategy_metrics_only_capability_v1`) must return Ok, and a
missing export, any other status or a failing call means no. A module that does not certify takes
actual curve capture and gets the same new document shape; `legacy` always uses the actual curve.
The two metrics ctypes binders and mirrors of the four receipt structs (32, 72, 24 and 232
bytes, size-checked at import) are optional like the existing reporting binders. When a receipt
reports that the run must be redone with the curve, the run is repeated once from the frozen
inputs and only that attempt's document is published; a native failure is forwarded unchanged.
The new option is refused together with an explicit report sink, a phase stream, selected-window
options, outputs, a benchmark run or symbol feeds, and it bypasses the implicit temporary sink so
a sink-capable module stays admissible.

Tests (docker/run_json_metrics_only_test.py, collected by the preflight pytest) extend the existing
sink fake library: parsing and every refusal, both profiles and both modes, key order and shared
raw bytes, receipt layout and proof rejection, the one-time replay and its cleanup order, frozen
inputs, native failures, the capability probe (Ok only; missing, failing and other statuses
refuse), and the absent-option output against the unchanged ordinary producer. The
memory-management page gains one paragraph.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ed and uncertified native modules

Two test native modules built through the production export macro share one small trading host
(fixed trade population, builtin report marks); only one specializes the metrics-plain declaration,
so the other is the control. A pytest file drives the real docker/run_json.py as a child process:

- default profile and jailed profile (inherited sealed arena): bare `--metrics-only`, explicit
  `--metrics-only=legacy` and no option, over a six-year tape; auto and legacy stdout must be
  byte-identical, the ordinary document must project onto the scalar document with exactly the
  three top-level omissions and the two arena counters, and the destruction-time receipt must show
  the route actually taken (streaming for the certified module, actual curve for the control);
- a Release-only memory row at 1,000,000 and 3,000,000 bars (three fresh processes per cell,
  wait4 peak RSS): the certified saving must be at least 12 bytes per bar and the uncertified
  control's below 4.

The module targets are declared in tests/CMakeLists.txt (the lane-file checker refuses target
statements in tests/cr150.d); the three ctest rows are registered in
tests/cr150.d/metrics-only-real.cmake and run through the no-skip pytest runner.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…n be accepted when annualized return is finite

The streaming metrics receipt carried the first and last timestamps of the run but not the last
equity mark, so the JSON runner had to refuse every curve-free attempt whose annualized return was
finite and rerun with the actual curve: a normal run never took the no-curve route. The receipt
now ends with the observed final mark (`actual_last_equity`) and a validity word
(`actual_last_equity_valid`, 1 exactly when a mark was observed). The receipt grows from 232 to 248
bytes (the containing result from 288 to 304); the extent, plan and stop-boundary PODs, the plan
comparison and the planned-view hash are untouched, so the observation never binds the plan.

The value is the fold's own held mark (streaming and replay), the authoritative curve's last point
(actual-curve branch, which clears an earlier held value first), the legacy curve walk's last point
or the legacy summary snapshot's last point; empty extents leave it invalid.

docker/run_json.py mirrors the field and the new size. A curve-free report with a finite annualized
return is accepted only with a valid finite final mark and the module's finite positive exported
initial capital (the first-mark fallback needs a real curve and is never invented). The document's
annualization diagnostic is then built by the existing helper from a two-row stand-in holding the
first timestamp and the last timestamp and equity, so there is one copy of the arithmetic.

Tests: the fake-library file (layout at independent offsets, finite-annualization acceptance with
sub-day, normal and overflow spans compared byte for byte with the legacy document, and refusals
for invalid validity words, nonfinite marks and missing or invalid capital); the C++ streaming,
native-module, monthly-replay and comparison tools compare the observation with an independently
executed actual-curve run on every route, including empty, stopped and recovery runs; the
real-process rows also compare the observed mark with the ordinary run's curve.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…he old name is accepted until the next minor

The reference-run readers (the range-end and first-bar lookups in run_strategy.py and the calendar
document in symbol_calendar.py) now take the range provenance dict from `rangeProvenance`, and from
the old member name only when the new one is absent or not a dict. One helper does the lookup for all
three readers, and the printed source label names the member that was actually read, so a tape that
still carries the old name prints the old label and a tape with the new name prints the new one.
Nothing else about the readers changes.

Tests: the four run_strategy test scripts use the new name; new rows cover a document with only the old
name (same results and old label for both readers and the calendar), both names present with different
values (the new name wins), and a new member that is not a dict next to an old one that is.
Docstrings and one test comment now use the new name.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…he sites it added

The metrics changes and the batch they sit on moved lines that documentation pages cite, and added
three sites the source-site gate indexes. Each stale citation was re-pointed by reading its sentence
against the current source (only the numbers inside the citations changed; every page keeps its line
count), and the gate data now describes the tree: three new manifest entries (unreviewed, none
bounded) for the final-month buffer allocations in the monthly replay and the assignment in the
metrics binding hash, the site records the tool prints for them, the identity of an existing
`<temp:string>` entry that moved scope, the unreviewed ceiling and the pinned open count (1483 -> 1486).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Adds the 1.8.0 entry for `--metrics-only[=auto|legacy]`, the optional capability export a module uses to
certify itself, the document shape, the refusals and the measured saving of 16 bytes of resident memory
per bar.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
sys.path.insert(0, str(ROOT / "tests"))

import run_json
import test_run_json_sink as ordinary
luisleo526 and others added 2 commits October 29, 2026 03:08
The three entries the metrics chain adds (the assignment in the metrics binding hash and the two
final-month buffer allocations in the monthly replay) move from U1580-U1582 to U1590-U1592, so they
cannot collide with the ids another change in flight takes on the same base. The entries, their
state (unreviewed), the site inventory, the ceilings and the pinned open count are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…e refusal list is not complete, record the temporary member name

Review changes for the metrics-only request:

- The changelog states the bound the acceptance test asserts (at least 12 bytes of resident memory per
  bar) instead of a figure the tree does not carry.
- The changelog and the memory-management page say the options the metrics-only request is refused
  with are not only the ones named (the report-shaping options are refused too).
- The changelog records that the reference-run scripts read the range-provenance member under its new
  name first and still accept its previous name until the next minor release.

The memory-management page replaces the sentence "Explicit sinks, phase transport, selected-window
options, outputs, benchmarks and symbol feeds are refused with this option." (introduced with the
option) by one that also names the other report-shaping options.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@luisleo526

Copy link
Copy Markdown
Collaborator Author

Folded into #632, the combined 1.8.0 batch-2 pull request. The branch is kept.

@luisleo526 luisleo526 closed this Oct 29, 2026
luisleo526 added a commit that referenced this pull request Oct 29, 2026
…ion, metrics-only mode (#632)

Combined second batch for 1.8.0. Each member was reviewed and proven on its own branch; this branch merges them onto main and is verified as one tree.

## Members (merged in this order)

- Keep a reversal's newly credited opening when its entry id had opposite-side history (fixes wrong positions, uncapped pyramiding and missing closed-trade rows after a reversal on a reused entry id; affected releases v1.1.0 through v1.7.0).
- Window buffers for technical-analysis series kept on main.
- Report channel contract and hosting jail budget contract (docs; #613, #614).
- Annualization diagnostic uses the engine's initial capital on every report path.
- Opt-in report replay companion (#607).
- Metrics-only run mode (#630).
- Comment-only rewording of shipped source comments.
- Source clones skip the in-repo `.ccache` directory, so a whole-tree copy no longer races a running compile.
- In-process iteration of the default-path report columns: `report["trades"]` and `report["equity_curve"]` iterate lazily from the report spool again, report their length and support integer indexing; iteration must happen before the report's resources close.
- CHANGELOG entries for the above, and re-pointed source-line citations in two documentation pages (line numbers only).
- The report-sink trade-identity test reference is re-recorded: the reversal fix changes the per-bar state hash input (open-position bookkeeping), while no trade or equity row changes; the CHANGELOG says so.

## Not in this batch

The live runner's report checkpoints (#598) and report accumulator (#617) move to the next minor release: testing showed a timing case that can leave a delivered order unacknowledged and a crash for modules whose handle is not an engine. Their pull requests stay open as drafts.

## Verification

- Native live suite (all native shards, thread and address sanitizer legs) on this tree's code.
- Full verify and the corpus parity sweep on this exact head.
- Every member was reviewed independently before it was merged here.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant