Skip to content

Commit 3e1e70d

Browse files
committed
CR-150 o1w-metrics round 1: equity metric folds, summary-first-pass mode (R01 R02 R05 R10), supervisor commit
1 parent 583c221 commit 3e1e70d

8 files changed

Lines changed: 1223 additions & 6 deletions

File tree

‎O1W_METRICS_REPORT.md‎

Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
R01/R02 equity implementation and R05 second-pass math are prepared for
2+
supervisor integration. R06 and R10 were already implemented by batch B.
3+
This is an uncommitted, unqualified lane diff, **not a completed SUMMARY_ONLY
4+
provider**. Terminal refold, selected capture, omitted trade rows, the versioned
5+
envelope and deterministic engine replay still need their owners' integration.
6+
No build, test, preflight, corpus or campaign result is claimed.
7+
8+
HEAD/base: `583c2217d2524e8f2bc4c8a77dfa336309d741fc`, branch `o1w/metrics`.
9+
No git add, commit or push. `git diff --stat <base>..HEAD` is empty because
10+
the supervisor commits this working diff. At handoff the three tracked files
11+
have 191 insertions / 6 deletions; the new header, two test units, lane CMake
12+
file and this report are additional untracked files. Use the working-tree
13+
diff, including those new files, for review.
14+
15+
**Rows and bounds**
16+
17+
| Row | Base behavior | Prepared change / resident bound | Evidence still required |
18+
| --- | --- | --- | --- |
19+
| R01 | `engine.hpp:587,1381`; `native_execution_consumer.cpp:13166` grows the ordinary sink-off curve. Batch B already holds only one point when scratch-backed sinking is enabled. | `record_equity_point` selects the summary branch by the active equity mode. One vector slot is reserved before the first callback; previous FULL capacity is released at run preparation. A fixed first-pass prefix excludes the editable held point. Snapshot/presentation replaces that point before folding it; no subtraction or regrouping. Fixed broker-extrema prefix is separately seeded from initial capital, while metric extrema seed from the FIRST equity point. | New allocation and math tests; terminal helper hook below; lab qualification. |
20+
| R02 | Ordinary `engine_report.cpp:272` allocates/copies `n` points. Batch-B ordinary sinking instead presents one point and rereads scratch. | Summary `fill_metrics_section` presents one stack point, finalizes fixed folds and returns `nullptr/0` resident curve. `equity_logical_point_count()` and `equity_summary_snapshot()` expose the separate logical count. There is no n-sized curve allocation, temporary curve, scratch writer or curve region in summary preparation. | Envelope must publish logical count with captured=0/external=0 and gate completion. R13 still owns removing trade arrays. |
21+
| R05 | `engine_metrics.cpp:54-125,383-414` already generates month/bar returns twice with fixed auxiliary state. | New pure `metrics_fold.hpp` provides the exact ordered first sums/counts and second variance/downside walks. Both series share one point replay and use final bar density. Finalizers return the existing 768-byte `pf_metrics_t`. Storage is `sizeof(EquityMetricsFold)` plus `sizeof(EquityMetricsSecondPass)` and fixed snapshots/results, independent of point/return/month counts. Sizes are printed by the new math test. | Raw-byte differential test, floating-environment review, actual engine replay by its owner. |
22+
| R10 | `TradeStatsAccumulator::State` and the percentage prefixes are already fixed. | No production change needed. `closed_trade_report_` is allocated at consumer line 3015, before `on_native_run_begin` at 3073; its ALL/LONG/SHORT pimpl allocations are fixed. `finish_streamed_trade_metrics:13549` copies three fixed states, then folds resident/terminal rows in order. Percentage prefixes remain scalar. | Existing independent accumulator test plus new allocation census: fold 1,000/100,000 rows with zero new allocations, compare fixed copy/finish allocation bytes and calls. |
23+
| R06 | Contrary to the older inventory, batch B includes `src/metrics_spill_curve.cpp`. | Already implemented: one array of 2,730 binary points (65,520 bytes), bounded positional refills, constructors/readers and both explicit `SpillCurve` metric instantiations. Untouched. Summary conflicts with a sequence sink and returns from preparation before creating a writer/curve region. | Run existing spill/scratch tests remotely. No PASS inferred from source presence. |
24+
25+
**Pure interface for R03/R04 and replay**
26+
27+
All types below are in `pineforge::metrics`; the new header includes no engine
28+
header. It accepts an externally supplied civil-month key, so a caller need
29+
not hold the process timezone mutex across engine dispatch.
30+
31+
- `equity_month_key(time_ms, chart_timezone)`: existing UTC month arithmetic,
32+
or a local-month lookup within its own short timezone guard. Same truncated
33+
whole-second convention as the real array path.
34+
- `EquityMetricsFold::fold(point, month_key, observation, exposed)`:
35+
closes the preceding held point and holds the next. Defaults are an ordinary
36+
observation and no exposure. For a selected anchor pass false/false.
37+
- `replace_last(point, month_key)`: rewrites only the held point, including its
38+
time/month/open P&L. Returns false when empty.
39+
- `first_pass()`: repeatable value snapshot; closes the held final month only
40+
in the returned copy. Includes logical/observation/exposure counts,
41+
first/last coordinates, metric extrema, return counts/sums and an ordered
42+
report-point witness. M=0 is represented by one non-observation anchor.
43+
- `EquityMetricsSecondPass(first_snapshot)`: fixes means, risk-free rates and
44+
density from pass A. Feed the same `fold`/`replace_last` interface in pass B.
45+
`finish()` is repeatable and returns `EquityRatioResult`.
46+
- `EquityRatioResult::matches(first_snapshot)`: checks this equity walk's
47+
count, endpoints, ordered point/month/classification witness and exact
48+
return sums/counts. Its `complete` flag is ONLY about this equity walk.
49+
It cannot replace the host's required trade/continuation/source witnesses.
50+
- `finalize_equity_metrics` / `finalize_metrics`: same guards, divisions,
51+
pow/sqrt expressions and order within each return sum as the array path.
52+
With no matching second result they produce explicitly first-pass-only
53+
fields; `needs_second_pass()` and the engine's readiness flag prevent an
54+
envelope owner from certifying such a result as complete.
55+
56+
Engine integration is intentionally an equity component, not a provider probe:
57+
`set_equity_report_mode`, `equity_report_mode`,
58+
`equity_logical_point_count`, `equity_summary_snapshot`,
59+
`equity_summary_ratios_ready`, `install_equity_ratio_result`, and the protected
60+
`refold_summary_equity_tail` hook. The mode choice takes effect on the next
61+
admitted run, not on an older report. Unknown modes and busy/conflicting
62+
configuration are refused without changing the pending equity mode.
63+
64+
**Required supervisor / sibling integration**
65+
66+
1. Merge the contract lane's enum/header/envelope declarations. This lane's
67+
`ReportMode` uses the exact FULL=0 / SUMMARY_ONLY=1 / STREAMED=2 words,
68+
values and uint32_t underlying type from its new document. Keep one enum;
69+
prefer the contract declaration using its public C constants. The two
70+
additions at the end of `ReportC` overlap mechanically.
71+
2. The versioned setter must establish exact replay/presentation eligibility
72+
and every sequence's omission policy before delegating to
73+
`set_equity_report_mode`. Do not advertise a summary-capable provider from
74+
this equity helper or its internal compilation macro. The envelope owner
75+
must bind `metadata.equity_curve.logical_count`, capture flags, current
76+
attempt ownership, and completion. Legacy report layouts are unchanged.
77+
3. At the START of `BacktestEngine::refold_equity_tail` in
78+
`src/native_execution_consumer.cpp:13227`, add the equivalent of
79+
`if (refold_summary_equity_tail()) return;`.
80+
Its current sink-off full-array walk sees only the held point in summary
81+
mode and would lose the earlier broker extrema. This is a required hook,
82+
not an optional optimization. I did not edit that out-of-scope function.
83+
4. R03/R04 must feed their own selected first pass, including the anchor and
84+
final presented observations, while preserving the existing validation
85+
order. The current selected/observed capture/read functions still use their
86+
arrays and bypass ordinary `fill_metrics_section`; they are untouched.
87+
Connect the pure fold at their authoritative point cadence before dropping
88+
those arrays. Freeze the validated summary and copy it on envelope reads,
89+
without another callback, presentation or replay.
90+
5. R13 must remove report trade arrays and populate the fixed trade blocks
91+
before equity finalization. For the current ordinary array producer, this
92+
lane reads its presented trade array once through existing accumulators.
93+
If an omitted-row producer supplies `total_trades>0`, it must already
94+
supply the ALL/LONG/SHORT blocks; the ALL count is checked. This lane does
95+
not claim O(1) report storage in T before that producer lands.
96+
6. Replay/HPO supplies finalized points to the second accumulator, validates
97+
all required trade/continuation/source witnesses and suppresses outward
98+
side effects. It can install a matching `EquityRatioResult` into ordinary
99+
equity assembly, or call the pure finalizer for selected capture. This
100+
lane implements no engine replay and does not fabricate its witnesses.
101+
102+
These integrations are outside the named function ranges. No native-consumer,
103+
source-layer, frozen-header, C ABI, transport or corpus file was edited.
104+
The additive equity setup in `begin_report_sink` is necessary to release old
105+
curve capacity and reserve the held point before callbacks, including sink OFF.
106+
107+
**Tests and fail-before status**
108+
109+
`test_metrics_fold` compares all 768 bytes to the REAL production array
110+
entry points, and pins those array results to the existing frozen
111+
`equity_stats_reference.hpp` / `trade_stats_reference.hpp`. First-pass-only
112+
comparison masks only the array oracle's four ratio fields. Full comparison
113+
uses the second accumulator. Fixtures include empty/one/two points, M=0
114+
anchor, zero/negative previous values, NaNs/infinity/signed zero, constant
115+
returns, zero/reversed spans, large alternating equity, month boundaries,
116+
UTC/Etc-UTC/New York/Taipei, deterministic random curves, terminal and
117+
presentation replacements crossing months, intermediate/repeated reads,
118+
partial second walks and reordered-point rejection.
119+
120+
Its intended first diagnostic against the frozen `583c2217d2` header closure
121+
is missing `pineforge/metrics_fold.hpp`. **Not executed; no recorded compiler
122+
diagnostic or fail-before receipt exists yet.**
123+
124+
`test_summary_metrics_allocations` replaces new/new[] (including aligned
125+
forms), measures report allocation calls and requested bytes, and asserts an
126+
identical result for 1,000 and 100,000 bars. It first verifies completed
127+
execution, callback count, every-bar ordered close/time witness, last
128+
timestamp, report bar count and zero-trade population. It checks held capacity,
129+
null resident curve, no scratch descriptor, logical count, ratio readiness and
130+
768-byte equality after feeding the FULL oracle's final points into the
131+
second accumulator. That feed is test-only; it is not engine replay. It also
132+
covers one/two points, presentation and FULL-to-summary handle reuse.
133+
134+
This allocation unit compiles against the frozen base without the new
135+
internal feature macro: its requested summary fixture then exercises the
136+
actual FULL path. Both populations still run; the null-curve/capacity and
137+
equal-report-byte assertions must fail, while its FULL control demonstrates
138+
the additional 99,000 * 24 curve bytes. This is a real growth RED, not a
139+
disabled/xfail row. **Not executed; actual RED and GREEN receipts are pending.**
140+
Its additional R10 census verifies zero allocation during trade folds and
141+
fixed copy/finish costs at both trade populations.
142+
143+
CTest names to run remotely:
144+
`test_metrics_fold`, `test_summary_metrics_allocations`, `test_metrics`,
145+
`test_metrics_l4d`, `test_metrics_point_source`, `test_trade_stats_accumulator`,
146+
`test_metrics_spill_curve`, `test_report_scratch`,
147+
`test_report_sink_scratch_curve`, and the selected-report suites after sibling
148+
integration. Required full preflight/release/debug/unit/ABI/corpus/parity work
149+
belongs to the supervisor's lab run on the integrated exact tree. No campaign
150+
sweep was run from this lane.
151+
152+
Local checks: git whitespace checks on tracked and new source/test/CMake
153+
files were clean. Only file reads, git reads and file edits were performed.
154+
There is no exposed lab/SM-CX execution tool in this session. Verification
155+
summary lines and log paths: **none; not run**.
156+
157+
**Hash, docs, files and ownership**
158+
159+
New report-only state is named explicitly: pending/active equity report mode,
160+
`summary_equity_fold_`, `summary_equity_snapshot_`, its validity/readiness
161+
flags, `summary_equity_ratios_`, and `summary_broker_equity_prefix_`.
162+
They are outside the broker-state region, like other output/capture state;
163+
no new broker hash leaf/domain/seed or waiver-file entry was introduced.
164+
The ordered point witness is a separate report/replay check. The broker
165+
prefix is reconstructible from the reported points and restores the existing
166+
four hashed extrema; it is not independent decision state. This output-only
167+
classification is explicit for supervisor hash review.
168+
169+
Files touched:
170+
`include/pineforge/engine.hpp`, `include/pineforge/metrics_fold.hpp` (new),
171+
`src/engine_metrics.cpp`, `src/engine_report.cpp`,
172+
`tests/test_metrics_fold.cpp` (new),
173+
`tests/test_summary_metrics_allocations.cpp` (new),
174+
`tests/cr150.d/core-cr150-o1w-metrics.cmake` (new),
175+
`O1W_METRICS_REPORT.md` (new).
176+
177+
Public documentation anchors shifted: none. PF_API declarations,
178+
implementations, symbol inventory/counts and legacy report/POD sizes changed:
179+
none. No dependency added. The contract file was read from the contract
180+
worktree and was not copied or edited here.
181+
182+
Sibling lanes must avoid edits to the new metrics header/tests/CMake lane
183+
file, `BacktestEngine::record_equity_point` and its summary members/helpers,
184+
ordinary `fill_metrics_section` and its equity preparation branch, and the new
185+
`equity_month_key` function. The supervisor must combine the one enum overlap
186+
and coordinate the explicit terminal-helper call above. Selected capture,
187+
envelope/transport, replay and omitted-row producers remain their owners'.

0 commit comments

Comments
 (0)