Skip to content

Commit 91fbd0e

Browse files
luisleo526claude
andcommitted
o1w-selected round 1: selected-capture final-value latches and presentation window (R03/R04 partial), supervisor commit
Executor result of lane core-cr150-o1w-selected (Codex gpt-6.1-sol), committed by the supervisor. Adds final-value latches and a fixed two-point presentation window, moves selected metric computation from report reads to capture. R03/R04 stay OPEN: runtime SUMMARY_ONLY mode, replay integration, the 100k/400k growth test and the whole-array hook rejection test are outstanding; the new unit depends on the metrics lane header. NOT VERIFIED: no compile, no fail-before run, no test, no corpus sweep was executed by the lane or by the supervisor on this commit. Lab rows to run: test_selected_capture_latches, test_selected_metrics_l17b, test_report_sink_selected_l17a, test_selected_window_report_integration, test_report_sink_equity, test_report_sink_scratch_curve, test_selected_production_routes, then full verification. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
1 parent 583c221 commit 91fbd0e

7 files changed

Lines changed: 1023 additions & 77 deletions

‎docs/design/o1-selected-capture.md‎

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
# O1 selected and observed capture handoff
2+
3+
Base: `583c2217d2524e8f2bc4c8a77dfa336309d741fc`.
4+
Rows: R03 and R04. This slice is groundwork plus capture-time metric freezing;
5+
it does **not** establish working SUMMARY_ONLY execution or close R03/R04.
6+
No remote fail-before, build, unit, corpus or charged-memory result has been
7+
obtained. No command requiring a local compiler or script was run.
8+
9+
## Runtime changes
10+
11+
At the base, `capture_presentation` copies selected M+1 or ordinary observed
12+
Bs points into its independently owned report unless a sink supplies scratch
13+
storage. `fill_selected_report` computes metrics again at each read, using
14+
the captured points but current engine metric inputs. Its sink-off array
15+
copy covers max(published count, metric count).
16+
17+
The capture now computes the complete existing `pf_metrics_t` after the one
18+
presentation hook and before publishing the capture. The trade blocks use
19+
the same closed-row prefix and final open-row order. Selected sink points
20+
are validated before metric computation, with the same typed errors and
21+
precedence as the existing final selected-sink scan. A report read copies the
22+
frozen metric block, with no metric source, trade fold or timezone walk.
23+
Removing a sink still explicitly requests the legacy full array; its
24+
materialization remains available and uses the independent captured source.
25+
26+
`SelectedCaptureLatches` separates final-value validation from the writer.
27+
Feed it final execution points in their recorded order. For a host that
28+
rewrites arbitrary earlier points, scan its final array at capture; do not
29+
latch provisional values. Its copyable scalar state preserves trade,
30+
open-lot, account, contiguity, then count refusal order. The existing selected
31+
sink validates its edited metric observations too, including a suffix hidden
32+
from publication; the anchor is excluded from observation validation.
33+
34+
`PresentationCaptureWindow` owns two points, two observation/exposure flag
35+
pairs, a logical count and a held count. `append_final` releases the oldest
36+
point directly to the existing scratch writer before reusing its slot.
37+
`append_metric_point` supplies the same release interface with the original
38+
observation and exposure flags for the forthcoming exact metric passes.
39+
`emit_metric_tail` visits every edited held point, including hidden points.
40+
`published_count` subtracts only the number hidden by the hook. None of these
41+
operations retains the released prefix, allocates a growing collection or
42+
introduces a population cap.
43+
44+
The resident state of these helpers is fixed by their types, independent of
45+
bars, observations, trades or keys. The frozen metric block reuses the
46+
existing 768-byte field in the capture. Existing scratch writers/readers each
47+
have a fixed 64 KiB window. FULL capture and FULL report arrays are still
48+
O(M) / O(Bs); scratch-backed STREAMED output remains a per-point external
49+
sequence. Neither qualifies as SUMMARY_ONLY.
50+
51+
Batch-B already supplied finalizable sink recording, L17a selected validation
52+
latches, the two-point selected presentation, L9s sink assembly and bounded
53+
scratch readers. Those pieces were extended rather than replaced. The
54+
range-end held-point rewrite and extrema prefix restore were already present;
55+
this lane did not change equity recording, sealing, digests or their headers.
56+
57+
## Exact metric integration boundary
58+
59+
The metrics lane's `metrics_fold.hpp` and `tests/test_metrics_fold.cpp` WIP
60+
specify the calls below. The new capture test includes that header and
61+
exercises both exact passes through the held window, comparing all 768 bytes
62+
to the frozen array oracle for 96 additional curves. That unit requires the
63+
supervisor to consume the metrics lane's header and implementation first;
64+
neither has been copied into this lane.
65+
66+
The expected runtime integration calls, already exercised by the new unit
67+
using that WIP's exact spellings, are:
68+
69+
1. Construct `metrics::EquityMetricsFold first` before execution.
70+
2. Insert the selected anchor with
71+
`window.append_metric_point(anchor, false, false, emit_first)`.
72+
Insert each final selected observation with its actual exposure flag;
73+
ordinary observed capture has no synthetic anchor.
74+
3. In `emit_first(point, observation, in_market)`, call
75+
`first.fold(point, metrics::equity_month_key(point.time_ms, chart_timezone),
76+
observation, in_market)`. The held terminal value is supplied only after
77+
execution rewrites, and the held presentation window only after its edit.
78+
Finish that window with `window.emit_metric_tail(emit_first)`.
79+
4. Take `const auto pass_a = first.first_pass()`. Bind
80+
`metrics::EquityMetricInputs{initial_capital, first_open, last_close,
81+
net_profit}` with the selected first/last input prices and M denominator.
82+
The anchor's observation flag is false, including M=0 and M=1.
83+
5. Construct `metrics::EquityMetricsSecondPass second(pass_a)` and walk one
84+
exact deterministic replay. Its visitor calls
85+
`second.fold(point, metrics::equity_month_key(point.time_ms, chart_timezone),
86+
observation, in_market)` over the same final presented points.
87+
6. Obtain `const auto ratios = second.finish()` and require
88+
`ratios.matches(pass_a)`. This is additional metric evidence; it does not
89+
replace the contract's ordered trade and final execution/source witnesses.
90+
7. Freeze `metrics::finalize_metrics(pass_a, inputs, all, longs, shorts,
91+
&ratios)` into the capture's existing metric block. A repeated report read
92+
performs none of these calls.
93+
94+
Do not infer exposure from nonzero open profit. Do not hold a non-recursive
95+
timezone guard across dispatch or replay. The recording lane must preserve
96+
the two still-editable points and their flags until presentation is final;
97+
folding the penultimate selected point before its possible edit is incorrect.
98+
99+
Still needed from the runtime integration owner: the stored/admitted
100+
SUMMARY_ONLY mode; its selected/observed capture branch and frozen logical
101+
counts; the pre-bar-zero point/replay provider; execution and trade witness
102+
gates; exact exposure flags; and output suppression. These require engine or
103+
consumer declarations outside this lane's assigned function ranges. No new
104+
per-run global registry, alternate mode API or implicit fallback was added.
105+
The integration owner must also update the read-time metric descriptions in
106+
`src/native_execution_consumer.hpp` and `include/pineforge/metrics_stream.hpp`;
107+
this lane did not edit those shared headers.
108+
109+
## Presentation hooks and compatibility
110+
111+
The inventory below comes from `git grep` for `present_report(` across
112+
include, src, tests and examples. C callback hook forwarding does not expose
113+
an additional whole-array presentation hook.
114+
115+
| Hook | Reads or edits |
116+
| --- | --- |
117+
| `BacktestEngine::present_report` in `include/pineforge/engine.hpp` | Default no-op. |
118+
| `PineStrategyHost::present_report` in `src/source/pine_strategy_host.cpp` | Final equity point; presented open-at-end trade rows in lot order; `total_trades`, `trades_len`, `net_profit` and closed-row PnL. It re-marks terminal rows, updates net PnL and rewrites the final point. It does not inspect equity history. |
119+
| `tests/test_report_sink_equity.cpp` fixture | Counts the presented length, delegates to Pine, edits the final point or shrinks the curve. |
120+
| `tests/test_report_sink_scratch_curve.cpp` fixture | Edits only the final point. |
121+
| `tests/test_report_sink_selected_l17a.cpp` fixture | Counts calls, reads/edits the last point and cuts one point; can throw. |
122+
| `tests/test_selected_metrics_l17b.cpp` `Host` | Reads presented trades and the window; edits its last two points. Its explicit legacy branch reads/edits the full curve's middle, last point and published length. It also snapshots engine history as a test oracle. |
123+
| Same file `LiveReportHost` | Counts calls and reads the scratch descriptor; no equity-array read. |
124+
| Same file `PineHost` | Counts calls and delegates to Pine's terminal presentation. |
125+
| Same file `InvalidWindowHost` | Deliberately replaces storage, grows/negates the length or throws after replacement. |
126+
| Same file `StorageHost` | Reads the presented first and last points and length. |
127+
| Same file `TradeStorageHost` | Reads/edits open-at-end trade rows, net PnL and final equity point. |
128+
| Same file `RandomCaptureHost` added here | Edits the last two points and hides zero, one or two points. |
129+
| `tests/test_selected_window_report_integration.cpp` fixture | Counts calls, throws or sets the published length to one. |
130+
| `tests/test_selected_production_routes.cpp` and `tests/test_selected_window_routes.cpp` fixtures | Record the presentation event; no array reads. |
131+
132+
FULL continues to invoke the full-array hook on the full-array route,
133+
including arbitrary middle edits and shrink/hide behavior. Existing sink
134+
hooks retain their bounded view. Summary mode must require an explicit
135+
bounded-presentation declaration and an exact replay provider before calling
136+
any custom hook. The conservative default for an arbitrary override is
137+
whole-array access; C++ cannot discover which indexes an override will read.
138+
139+
For a whole-array-only presenter, summary negotiation must return
140+
`PF_REPORT_MODE_REPLAY_UNAVAILABLE` before execution, as specified by the
141+
contract lane's `docs/design/o1-report-contract.md`, “Exact second replay for
142+
SUMMARY_ONLY.”
143+
The provider/transport must surface the requested mode's documented error
144+
rather than invoke the hook with a shorter array. This declaration/error
145+
boundary and the required successful-summary-versus-FULL hook test remain
146+
unimplemented here; a documentation entry is not runtime coverage.
147+
148+
## Verification and remaining acceptance
149+
150+
No tests in this slice have been executed. Expected fail-before behavior:
151+
152+
| Test | Expected failure on the base |
153+
| --- | --- |
154+
| `test_selected_metrics_l17b` / `captured_metrics_are_frozen` | Repeated reads recompute percentages/exposure/CAGR from deliberately changed execution metric inputs, changing the raw metric block. |
155+
| Same executable / `frozen_sink_reads_need_no_metric_storage` | A selected read allocates a `SpillCurve` reader and trade folds; an observed read allocates a scratch reader and trade folds. The allocation-refusal interval detects both. |
156+
| `test_selected_capture_latches` | The frozen previous-epoch header closure lacks the new latch/window and metric-fold headers. The test also compares all failure combinations and 2,048 random arrays against the independent original scan. No remote first diagnostic is recorded yet. |
157+
158+
The added random engine captures assert actual window-script counts, compare
159+
all 768 metric bytes to frozen array reference functions across 72 curves
160+
including M=0/M=1 and three timezones, and compare every published sink point.
161+
The helper tests assert 100,000 and 400,000 consumed points before checking
162+
allocation counts. These helper populations are **not** the requested real
163+
script-bar SUMMARY_ONLY growth test. The inherited selected sink fixture
164+
asserts real M at 1,000 and 1,000,000 bars; it is already bounded in Batch-B.
165+
166+
Outstanding: real SUMMARY_ONLY capture at M=100,000 and M=400,000 with a
167+
growing-before/fixed-after receipt; whole-array presenter negotiation in
168+
SUMMARY_ONLY and successful FULL behavior; complete B1 replay/witness
169+
integration; and all remote verification. Do not promote R03 or R04 to a
170+
complete O(1) verdict from this slice.
171+
172+
Lab CTest rows to run are `test_selected_capture_latches`,
173+
`test_selected_metrics_l17b`, `test_report_sink_selected_l17a`,
174+
`test_selected_window_report_integration`, `test_report_sink_equity`,
175+
`test_report_sink_scratch_curve` and `test_selected_production_routes`.
176+
The supervisor must also run preflight, complete all-target release/debug
177+
verification and the full corpus on the final integrated tree.
178+
179+
No ABI exports, declaration counts, existing document anchors or hash leaves
180+
were changed. The new latches/window are local report state, not new durable
181+
execution state. The existing report-only metric field remains under its
182+
existing output-state waiver. Other lanes must preserve the bodies of
183+
`capture_presentation` and `fill_selected_report`, the two new internal
184+
headers, this document, the new test unit/registration and the added capture
185+
tests when merging. The only edits outside the owned function bodies are
186+
their two include directives; no other lane's function or header was edited.

0 commit comments

Comments
 (0)