Repository navigation
run_json: --metrics-only, a certified module takes the no-curve metrics route - #630
Closed
luisleo526 wants to merge 12 commits into
Closed
luisleo526 wants to merge 12 commits into
luisleo526 wants to merge 12 commits into
Conversation
… no-curve run covers multi-month histories The ordered return replay removed the per-bar equity curve only for a run whose need mask asked the bar Sharpe and Sortino alone and whose history stayed inside one calendar month: with the curve dropped, a second month sent the run back through the curve (whole-report recovery). The monthly Sharpe and Sortino now come from bounded state: the equity at each closed month's end (at most 4,096 closed months, plus the current and held month's scalars; about 32 KiB steady, 48 KiB while the array grows), folded with the legacy operation order over the per-month returns, so the values are the legacy values bit for bit. - The largest no-curve mask is now every need but Returns and Curve (0x37). The default mask still asks Returns and Curve and keeps the curve; no default and no public struct size changes. - The two-month recovery trigger fires only when the monthly replay did not complete: a larger month population than the bound, a reversed month key, an allocation failure, an interrupted run, or a planned endpoint outside the range the month key computes in without overflow (those keep the curve). - The month-key helper in the engine header guards the speculative month-key lookups the recorders make before the monthly fold can refuse a timestamp; no member layout or virtual slot changes. The month key's own integer arithmetic is not changed here. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
New test_metrics_monthly_replay (78 rows) compares the monthly-replay route with the legacy route computed from the real actual curve in the same binary (streaming switch off), bit for bit and for every field of the equity summary, with an independent reference where the neighbouring rows have one: generated series of 1, 2, 3, 12 and 60 months, month boundaries at the exact last and first bar, a one-bar month, a chart timezone whose local month differs from the UTC month, flat months, gaps of empty months, negative, zero and heavy-tail months, the Sharpe-only, Sortino-only and both masks and the largest no-curve mask, stopped runs, interrupted execution keeping the whole-report recovery, a month population above the bound, and the default mask keeping every requested curve point. Registration through its own tests/cr150.d lane file. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…one capability export A loaded native module can now opt in to the no-curve metrics route. A host certifies itself by specialising `native_module::MetricsPlainHostV1<Host>` (exact type: a derived host does not inherit it) with its truthful `MetricsPlainDeclarationV1`; the generic macro declares no host plain on its own. When the engine's metrics preference is On and the host is certified, the run wrapper derives the planned extent and the plain-run inputs from the invocation (count, first and last bar time, the staged native spec, the timeframe arguments, report policy, subscriptions and feeds), binds the input, spec and view with fixed-width digests, and calls the engine's plain-run admission and preparation immediately before the run. Anything else (preference Off, an uncertified host, a throwing or unknown declaration, a non-plain answer) leaves the engine on the actual-curve route exactly as before, and the block never throws. New optional C export `strategy_metrics_only_capability_v1()`, generated for every module: it returns Ok (0) for a certified host and NotImplemented (-1) otherwise, so a loader probes by calling it. No existing export, struct size, default or output changes. The runtime export inventory check pins the generated module surface separately from the static runtime exports, and the stability page documents the probe, the specialisation shape and the recovery contract. Sentence replaced in docs/pages/abi-stability.md, named for the revert check: "A built strategy `.so` exposes 110 compiled-strategy `PF_API` declarations (94 runtime implementations plus sixteen generated exports) plus 43 native-" (the stability page now states the aggregated inventory counts and lists the optional module export separately; introduced by 95f4fd1). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ts, on and off) New test_native_module_metrics_only builds two small concrete hosts through the real module macro, one certified and one not, over a 48-bar, twelve-month feed with long and short trades, fees and a terminal open position. Rows: the capability export returns Ok and NotImplemented for the two hosts and a subclass cannot inherit certification; a certified host with the preference On reaches Streaming through both generated run entries with every scalar metric word-identical to an independent actual-curve control (and no curve retained); an uncertified host, the Off preference and the default all-needs mask keep the actual curve and the same words; a selected window and a synthesised intrabar path are not plain and keep the curve; a throwing or unknown-version declaration cannot escape or admit; the input, spec and view identities are stable for frozen input and change with an interior volume word or the fee setting; a stopped run keeps the whole-run recovery receipt. Registered through its own tests/cr150.d file. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ute, with whole-run recovery New option `--metrics-only[=auto|legacy]` (a bare option means auto). The document it prints is the ordinary document without `trades`, `equity_curve`, `elapsed_seconds` and the two scratch arena byte counters; every other key keeps its name, value and relative order. Without the option nothing changes: requests, errors, checks and defaults are byte for byte what they were. `auto` asks the engine for the streaming preference (on, need mask: Sharpe, Sortino, monthly ratios, equity summary, trade statistics) only when the loaded module certifies itself: the module's own capability export (`strategy_metrics_only_capability_v1`) must return Ok, and a missing export, any other status or a failing call means no. A module that does not certify takes actual curve capture and gets the same new document shape; `legacy` always uses the actual curve. The two metrics ctypes binders and mirrors of the four receipt structs (32, 72, 24 and 232 bytes, size-checked at import) are optional like the existing reporting binders. When a receipt reports that the run must be redone with the curve, the run is repeated once from the frozen inputs and only that attempt's document is published; a native failure is forwarded unchanged. The new option is refused together with an explicit report sink, a phase stream, selected-window options, outputs, a benchmark run or symbol feeds, and it bypasses the implicit temporary sink so a sink-capable module stays admissible. Tests (docker/run_json_metrics_only_test.py, collected by the preflight pytest) extend the existing sink fake library: parsing and every refusal, both profiles and both modes, key order and shared raw bytes, receipt layout and proof rejection, the one-time replay and its cleanup order, frozen inputs, native failures, the capability probe (Ok only; missing, failing and other statuses refuse), and the absent-option output against the unchanged ordinary producer. The memory-management page gains one paragraph. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ed and uncertified native modules Two test native modules built through the production export macro share one small trading host (fixed trade population, builtin report marks); only one specializes the metrics-plain declaration, so the other is the control. A pytest file drives the real docker/run_json.py as a child process: - default profile and jailed profile (inherited sealed arena): bare `--metrics-only`, explicit `--metrics-only=legacy` and no option, over a six-year tape; auto and legacy stdout must be byte-identical, the ordinary document must project onto the scalar document with exactly the three top-level omissions and the two arena counters, and the destruction-time receipt must show the route actually taken (streaming for the certified module, actual curve for the control); - a Release-only memory row at 1,000,000 and 3,000,000 bars (three fresh processes per cell, wait4 peak RSS): the certified saving must be at least 12 bytes per bar and the uncertified control's below 4. The module targets are declared in tests/CMakeLists.txt (the lane-file checker refuses target statements in tests/cr150.d); the three ctest rows are registered in tests/cr150.d/metrics-only-real.cmake and run through the no-skip pytest runner. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…n be accepted when annualized return is finite The streaming metrics receipt carried the first and last timestamps of the run but not the last equity mark, so the JSON runner had to refuse every curve-free attempt whose annualized return was finite and rerun with the actual curve: a normal run never took the no-curve route. The receipt now ends with the observed final mark (`actual_last_equity`) and a validity word (`actual_last_equity_valid`, 1 exactly when a mark was observed). The receipt grows from 232 to 248 bytes (the containing result from 288 to 304); the extent, plan and stop-boundary PODs, the plan comparison and the planned-view hash are untouched, so the observation never binds the plan. The value is the fold's own held mark (streaming and replay), the authoritative curve's last point (actual-curve branch, which clears an earlier held value first), the legacy curve walk's last point or the legacy summary snapshot's last point; empty extents leave it invalid. docker/run_json.py mirrors the field and the new size. A curve-free report with a finite annualized return is accepted only with a valid finite final mark and the module's finite positive exported initial capital (the first-mark fallback needs a real curve and is never invented). The document's annualization diagnostic is then built by the existing helper from a two-row stand-in holding the first timestamp and the last timestamp and equity, so there is one copy of the arithmetic. Tests: the fake-library file (layout at independent offsets, finite-annualization acceptance with sub-day, normal and overflow spans compared byte for byte with the legacy document, and refusals for invalid validity words, nonfinite marks and missing or invalid capital); the C++ streaming, native-module, monthly-replay and comparison tools compare the observation with an independently executed actual-curve run on every route, including empty, stopped and recovery runs; the real-process rows also compare the observed mark with the ordinary run's curve. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…he old name is accepted until the next minor The reference-run readers (the range-end and first-bar lookups in run_strategy.py and the calendar document in symbol_calendar.py) now take the range provenance dict from `rangeProvenance`, and from the old member name only when the new one is absent or not a dict. One helper does the lookup for all three readers, and the printed source label names the member that was actually read, so a tape that still carries the old name prints the old label and a tape with the new name prints the new one. Nothing else about the readers changes. Tests: the four run_strategy test scripts use the new name; new rows cover a document with only the old name (same results and old label for both readers and the calendar), both names present with different values (the new name wins), and a new member that is not a dict next to an old one that is. Docstrings and one test comment now use the new name. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…he sites it added The metrics changes and the batch they sit on moved lines that documentation pages cite, and added three sites the source-site gate indexes. Each stale citation was re-pointed by reading its sentence against the current source (only the numbers inside the citations changed; every page keeps its line count), and the gate data now describes the tree: three new manifest entries (unreviewed, none bounded) for the final-month buffer allocations in the monthly replay and the assignment in the metrics binding hash, the site records the tool prints for them, the identity of an existing `<temp:string>` entry that moved scope, the unreviewed ceiling and the pinned open count (1483 -> 1486). Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Adds the 1.8.0 entry for `--metrics-only[=auto|legacy]`, the optional capability export a module uses to certify itself, the document shape, the refusals and the measured saving of 16 bytes of resident memory per bar. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
| sys.path.insert(0, str(ROOT / "tests")) | ||
|
|
||
| import run_json | ||
| import test_run_json_sink as ordinary |
The three entries the metrics chain adds (the assignment in the metrics binding hash and the two final-month buffer allocations in the monthly replay) move from U1580-U1582 to U1590-U1592, so they cannot collide with the ids another change in flight takes on the same base. The entries, their state (unreviewed), the site inventory, the ceilings and the pinned open count are unchanged. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…e refusal list is not complete, record the temporary member name Review changes for the metrics-only request: - The changelog states the bound the acceptance test asserts (at least 12 bytes of resident memory per bar) instead of a figure the tree does not carry. - The changelog and the memory-management page say the options the metrics-only request is refused with are not only the ones named (the report-shaping options are refused too). - The changelog records that the reference-run scripts read the range-provenance member under its new name first and still accept its previous name until the next minor release. The memory-management page replaces the sentence "Explicit sinks, phase transport, selected-window options, outputs, benchmarks and symbol feeds are refused with this option." (introduced with the option) by one that also names the other report-shaping options. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Collaborator
Author
|
Folded into #632, the combined 1.8.0 batch-2 pull request. The branch is kept. |
luisleo526
added a commit
that referenced
this pull request
Oct 29, 2026
…ion, metrics-only mode (#632) Combined second batch for 1.8.0. Each member was reviewed and proven on its own branch; this branch merges them onto main and is verified as one tree. ## Members (merged in this order) - Keep a reversal's newly credited opening when its entry id had opposite-side history (fixes wrong positions, uncapped pyramiding and missing closed-trade rows after a reversal on a reused entry id; affected releases v1.1.0 through v1.7.0). - Window buffers for technical-analysis series kept on main. - Report channel contract and hosting jail budget contract (docs; #613, #614). - Annualization diagnostic uses the engine's initial capital on every report path. - Opt-in report replay companion (#607). - Metrics-only run mode (#630). - Comment-only rewording of shipped source comments. - Source clones skip the in-repo `.ccache` directory, so a whole-tree copy no longer races a running compile. - In-process iteration of the default-path report columns: `report["trades"]` and `report["equity_curve"]` iterate lazily from the report spool again, report their length and support integer indexing; iteration must happen before the report's resources close. - CHANGELOG entries for the above, and re-pointed source-line citations in two documentation pages (line numbers only). - The report-sink trade-identity test reference is re-recorded: the reversal fix changes the per-bar state hash input (open-position bookkeeping), while no trade or equity row changes; the CHANGELOG says so. ## Not in this batch The live runner's report checkpoints (#598) and report accumulator (#617) move to the next minor release: testing showed a timing case that can leave a delivered order unacknowledged and a crash for modules whose handle is not an engine. Their pull requests stay open as drafts. ## Verification - Native live suite (all native shards, thread and address sanitizer legs) on this tree's code. - Full verify and the corpus parity sweep on this exact head. - Every member was reviewed independently before it was merged here. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
run_jsongains--metrics-only[=auto|legacy](a bare option meansauto). The document it prints is the ordinary document withouttrades,equity_curve,elapsed_secondsand the two scratch-arena byte counters; every other key keeps its name, value and relative order. Without the option nothing changes.With
auto, a native module that certifies itself takes the engine's ordered-replay metrics route, so a run no longer keeps the full per-bar equity curve: Sharpe, Sortino, the monthly ratios, the equity summary and the trade statistics are computed as the run goes. The document is byte for byte the onelegacyprints. Commits, in order:strategy_metrics_only_capability_v1, reports it (0 certified, -1 not). The ABI inventory and page list it. A host that does not specialize is never treated as plain.run_json: option parsing and refusals (an explicit report sink, a phase stream, selected-window options, outputs, a benchmark run, symbol feeds), the capability probe (only status 0 counts; a missing export, any other status or a failing call means no), mirrors of the receipt structs (size-checked at import), whole-run recovery (one repeat from the frozen inputs when a receipt says the curve is needed), and the document shape.actual_last_equityand a validity word): the receipt grows from 232 to 248 bytes, appended; plan, extent and stop structs, the plan comparison and the admitted-view hash are untouched.run_jsonaccepts a curve-free result with a finite annualized return only with a valid finite final mark and the module's exported initial capital, and builds the annualization diagnostic with the existing helper from a two-row stand-in curve (one copy of the arithmetic).docker/run_json.pyin the default and jailed profiles over a six-year tape: auto and legacy stdout are byte-identical, the ordinary document projects onto the scalar document, the destruction-time receipt shows the route actually taken, and a Release-only memory row (1,000,000 and 3,000,000 bars, three fresh processes per cell) requires a saving of at least 12 bytes per bar for the certified module and under 4 for the control.rangeProvenancefirst; the old member name is accepted until the next minor.Verification
Measured on CLOUD: the real-process rows failed before the receipt field existed (two destroyed handles, 0.06 bytes per bar saved) and pass after; mutants that make the capability probe always false or always true turn the certified and the control rows red respectively; the range-provenance mutants (fallback deleted, new name ignored) turn their rows red. Full legs on the rebased code (release 1406 tests, debug 1311, sanitizers, kernel, parity 312 of 312 probes and the subset), and preflight 59 of 59 stages.
🤖 Generated with Claude Code