Skip to content

Commit f736676

Browse files
luisleo526claude
andauthored
Add selected native closes and synchronous current-point execution (#250)
* Add selected native closes and synchronous current-point execution * Respect the decision floor for current execution and retain v13 CI proof * Recognize exact engine RTTI in sanitized ABI epoch rejection * Derive ABI provider epoch from archive symbols, not the --library role The sanitized cross-epoch RTTI allowance inferred the provider's engine epoch from whether the runtime path equalled --library. In --old-rejections-only mode the authenticated e60 v13 archive may itself be supplied as --library, so a current v14 caller's expected rejection was misclassified as same-epoch and its exact-owner RTTI refused (false failure, never a false acceptance). Read the epoch from the archive's defined BacktestEngine symbols instead, memoized per archive, and keep the allowance decision in one tested predicate: exact caller-owner RTTI is tolerated only for a sanitized cross-epoch negative link. Full-matrix behaviour is unchanged. Add a tooling test for symbol-derived epochs (none/several refused) and for the allowance predicate across both directions and sanitizer states. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_015wvgEP9WUyEQhta7ZDqUB4 * Fence every epoch-14 engine shape against the v13 providers and close review gaps The settlement ABI checker skipped the engine storage-declaration, virtual- inventory and most layout-word comparisons whenever the provider epoch differed from the current one. With every real provider at epoch 13 that bypass would have let any later epoch-14 commit change BacktestEngine storage, its vtable or the compared aggregate layouts without an epoch bump, even though all of those compare equal today. Compare them unconditionally: storage declarations, the virtual inventory and every emitted layout word must match the v13 providers; a mismatch names the differing words. The four native headers that the v14 slice deliberately changed are the only enumerated exemptions for the v13->v14 transition, each recorded in the receipt with both digests and pinned to its reviewed bytes, so a further change within v14 fails until the epoch is bumped and the pin re-recorded. Hoist the per-link decision into link_outcome() with a really memoized provider_engine_for(), and record the provider epoch on every link row. Tests: three tooling tests drive the review-2 --old-rejections-only alias configuration through the production link decision (memo reads once per archive), the header exemption matrix and the full layout comparison, plus one for the exempted-header pins. The waiting-owner current-execution refusal is asserted exactly through both preview and execute_current, and the resting replay contract gains a selected BindOpenings close scenario (cohort of two, unrelated opening untouched, cancelled cohort close carrying OpeningsClose authority) compared across two hosts at every boundary. Docs: the settlement ABI fixture guide lists all three real providers, the --v13-receipt flag and what the v13->v14 comparison actually checks; abi-stability.md states the exemption and pin rules. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_013y1CvsXqLFMRgAW7TXJQMs --------- Co-authored-by: Claude Fable 5.1 <[email protected]>
1 parent c3ed455 commit f736676

49 files changed

Lines changed: 3707 additions & 508 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/ci.md‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,11 @@ for different profiles and toolchains. The verifier never deletes a build tree
4343
or replaces an incompatible historical ABI receipt. Use a fresh directory when
4444
its configuration no longer matches.
4545

46+
Run Release, Debug and sanitizer profiles sequentially in one checkout, or use
47+
separate worktrees for parallel runs. Their tutorial targets write shared `.so`
48+
files under `tutorial/`, even with separate build directories. Hosted CI jobs
49+
already use separate checkouts.
50+
4651
```sh
4752
python3 scripts/ci_verify.py debug --jobs 4
4853
python3 scripts/ci_verify.py sanitizers --jobs 4
@@ -70,11 +75,12 @@ separate. Git tags and checkout depth cannot change the smoke-test expectation.
7075
Ordinary CMake builds retain `AUTO`, the existing git-describe-first behavior.
7176
No release tag or VERSION value is rewritten by verification.
7277

73-
The verifier fetches the pinned historical ABI commits `e60e571` (R2) and
74-
`0e18690` (selected settlement, before exact reversal) without tags only when
75-
each object is missing. It builds both full historical static libraries with
76-
tests disabled, or validates and reuses their matching prepared receipts under
77-
`settlement-abi-base/` and `settlement-abi-prior/`. Compiler, configuration and
78+
The verifier fetches the pinned historical ABI commits `e60e571` (R2),
79+
`0e18690` (selected settlement, before exact reversal), and `c3ed455` (native
80+
host v13) without tags only when each object is missing. It builds all three
81+
full historical static libraries with tests disabled, or validates and reuses
82+
their matching prepared receipts under `settlement-abi-base/`,
83+
`settlement-abi-prior/`, and `native-abi-v13/`. Compiler, configuration and
7884
version-source mismatches refuse reuse without deleting the old evidence.
7985
Each profile needs matching providers; a Mac Release archive cannot replace
8086
a Linux sanitizer build. CTest itself performs no network fetch.

‎docs/pages/abi-stability.md‎

Lines changed: 58 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -112,20 +112,57 @@ notice:
112112
- The shape of internal log lines (use them for humans, not parsers).
113113

114114
Rebuild generated and native C++ objects against matching engine headers and
115-
runtime. Resting native request values and expanded host event snapshots
116-
advance `PendingOrder`, `BacktestEngine`, `NativeStrategyHost`, and the private
117-
native consumer to `engine_script_run_v13`. Exact pre-change e7d023d/v12
118-
headers (and the e7d023d `native_order_v1` closure) are authenticated before
119-
native, generated-style, standalone PendingOrder, and return-only
120-
`native_events()` pairing checks. A v12 host that only calls `native_events()`
121-
cannot silently decode a later `CommandEvent` layout: the method lives on the
122-
v13 class even if `sizeof(NativeStrategyHost)` is unchanged. Allocation-free
123-
`NativeFailureContext` (`NativeInRunCause` / `NativeInRunRecipient` /
124-
`NativeInRunCursor`) is part of that same epoch13 host record; it does not
125-
open a C ABI or new engine epoch. Current/old
126-
matching links must succeed and stale pairings must fail for the expected
127-
qualified symbols. Earlier v2–v10 and v12 controls remain; every translation
128-
unit must compile before any mismatch is accepted. No pairing executable runs.
115+
runtime. R4-A selected/current execution advances `PendingOrder`,
116+
`BacktestEngine`, `NativeStrategyHost`, and the private consumer to
117+
`engine_script_run_v14`. The host capability macro is
118+
`PINEFORGE_HAS_NATIVE_STRATEGY_HOST_V14`. Native order/core/event values are
119+
`native_order_v3`; driver types are `native_driver_v4`.
120+
121+
The authenticated c3ed455 epoch-13 header closure is tracked under
122+
`tests/fixtures/native_cpp_abi/host-c3ed455`. The verifier prepares a full real
123+
archive from that immutable source with the current profile's compiler and
124+
settings. Constructor/vtable, return-only `native_events()`, host observation,
125+
core request, driver and current-execution method callers compile before links
126+
are interpreted. Old-old and new-new controls link; cross-epoch callers reject
127+
at the expected namespace. No ABI caller executable is run.
128+
129+
The real e60 R2 and 0e R3 providers remain mandatory. Their former epoch-13 to
130+
current positive pairs are now explicit epoch rejections, with historical
131+
old-old and current-current sanity retained.
132+
133+
The `engine_script_run_v13` to `engine_script_run_v14` transition drops no
134+
check. Against each epoch-13 provider the checker still compares, in full and
135+
unconditionally:
136+
137+
* every engine named data declaration in source order (252 declarations, 251
138+
of them non-static data members) and the entire virtual method inventory —
139+
an epoch transition is never a licence to change engine storage or the
140+
vtable;
141+
* every compiler-emitted layout word — all 789 against e60 R2 and all 793
142+
against 0e R3, covering `sizeof`/`alignof` of `BacktestEngine`,
143+
`PendingOrder`, the native aggregates and the selected/projection types, plus
144+
the offset/size/alignment triple of each of the 251 engine data members, not
145+
only the leading financial `Result`/`SettlementInspection`, status and
146+
Action/CloseScope words. The receipt's `layout.comparedWords` and
147+
`priorLayout.comparedWords` therefore equal their `wordCount`;
148+
* every frozen native header's text, with exactly four enumerated exemptions —
149+
`native_order.hpp`, `native_host.hpp`, `market_driver.hpp` and
150+
`execution_consumer.hpp`, the headers that legitimately advance with
151+
`native_order_v3`, host v14, `native_driver_v4` and consumer v5. Each actual
152+
difference is recorded in `frozenShape.exemptedHeaders` with both digests and
153+
its transition; an exempted header that did not change records nothing, and
154+
any other differing header still raises. The exemption table lives in one
155+
module constant keyed by the exact transition it belongs to, so a future
156+
v14→v15 transition must be added explicitly rather than inherited.
157+
158+
`native_order_identity.hpp`, `native_run_spec.hpp` and `native_calendar.hpp`
159+
stay byte-frozen across the transition. Once a frozen v14 provider is added
160+
after merge, that provider's pairing carries no exemptions at all and the
161+
header-text fence is restored for those four headers too.
162+
163+
Earlier v2–v10 and v12 controls remain. Reusing an uninstrumented historical
164+
Release archive in a sanitizer profile is refused; preparation never overwrites
165+
an existing provider directory or substitutes a symbol stub for a real archive.
129166

130167
New standalone lifecycle values and `Lifecycle` own the inline namespace
131168
`pineforge::exit_legs::lifecycle_v1`; new admission values, `Draft`, `Journal`
@@ -165,17 +202,19 @@ its creating strategy module. A fully self-contained old module can still use
165202
its own matching runtime; this check does not turn it into a v11 module.
166203

167204
The integrated representation advances the broker fingerprint domain to
168-
`pineforge-broker-state/v13` and stream fingerprint version to 13. Native
169-
consumer identity is `native-consumer/v4`; driver v3 shapes, `close_scope_v1`,
205+
`pineforge-broker-state/v14` and stream fingerprint version to 14. Native
206+
consumer identity is `native-consumer/v5`; driver v4 appends CurrentExecution=8, while `close_scope_v1`,
170207
and `native_run_spec_v1` stay. Stable `RunIdentity` / `RequestHandle` / `Birth`
171208
remain `native_order_v1`; new request, core, and event values own
172-
`native_order_v2`. The original admission observation and prior-book direction
209+
`native_order_v3`. The original admission observation and prior-book direction
173210
are hashed as canonical facts; the three derived placement views add no
174211
redundant folds. Lifecycle definitions, generations, obligations and replay
175212
receipts, plus causal journal state remain represented. Existing
176213
reservation, Pine instruction, activation, quantity, predecessor and birth facts
177-
remain represented. The Pine component schema remains 1; it is
178-
independent of the aggregate fingerprint version. Prior v2–v12 fingerprints are
214+
remain represented. Selected cohorts, their executed live scopes, current callback quote/cutoff facts
215+
and queued notification order contribute to native continuation identity.
216+
The Pine component schema remains 1; it is
217+
independent of the aggregate fingerprint version. Prior v2–v13 fingerprints are
179218
not comparable. Fingerprints are replay checks, not serialized checkpoints or
180219
complete hashes of private strategy state. The native runner already binds
181220
its strategy-library SHA; its ledger format and Python provenance fingerprints

‎docs/pages/native-engine.md‎

Lines changed: 75 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,7 @@ Serialized external C++ calls may command only **between realtime inputs**,
149149
never reentrantly during input processing. There is no C request API in this
150150
slice.
151151
152-
`native_order::Request` values belong to `native_order_v2`; identity types stay
152+
`native_order::Request` values belong to `native_order_v3`; identity types stay
153153
`native_order_v1`. Label/comment remain inert text. The market default path
154154
still constructs from:
155155
@@ -165,6 +165,79 @@ cycle identity through scoped settlement. Group cancellation/reduction is
165165
caused by committed execution events. See the request header for the exact
166166
value types; source-specific Pine lowering remains codegen/adapter work.
167167
168+
### Selected exposure and current execution (R4-A)
169+
170+
`BindOpenings{{first, last}, cycle}` on a Flatten or explicit-unit Reduce binds
171+
one fixed cohort of already-live opening provenances. All handles must be
172+
unique, nonzero, in this run and physically live in the supplied positive
173+
cycle. Enrollment rejects the entire request if any member is invalid.
174+
Accepted definitions use canonical handle order; execution still closes lots
175+
in physical FIFO order. Later retirement narrows the live subset, while the
176+
immutable definition retains the original cohort. Several fragments of one
177+
opening share its provenance. A new handle with the same label is excluded.
178+
179+
One selected request produces one execution and ticket, even when it closes
180+
several rows. `ExecutionAppliedEvent::scope` uses native `ExecutionScope`
181+
(Book, OpeningExposure, SelectedExposure). Financial `execution::CloseScope`
182+
remains the original two-alternative type. The event's committed row range
183+
identifies actual contributors to a quantity-limited close.
184+
185+
Inside `on_native_bar` or `on_native_applied`, a request accepted or replaced
186+
in that exact callback can be consumed synchronously:
187+
188+
```cpp
189+
native_order::Request request{native_order::Flatten{}, "close cohort", ""};
190+
request.owner = native_order::BindOpenings{{first, last}, cycle};
191+
auto accepted = submit(request);
192+
if (accepted.handle) {
193+
NativeCurrentExecution command{*accepted.handle, NativeCurrentPriceRule::NearestTick};
194+
auto preview = inspect_current_execution(command);
195+
// The adapter may validate its own policy here. Preview is never an apply token.
196+
if (!preview.refusal && preview.settlement_readiness == execution::Status::Applied) {
197+
auto outcome = execute_current(command);
198+
// Applied effects and relationship drains are visible before this statement.
199+
}
200+
}
201+
```
202+
203+
The command contains only a target and price rule. Membership comes from the
204+
request. It cannot supply another selection, a saved cursor, price, ticket,
205+
or prepared financial plan. Current execution supports Market with
206+
ImmediateRemaining, resolved units, Independent/BindOpening/BindOpenings
207+
ownership and existing groups; current Transact remains Independent.
208+
Priced, budgeted and waiting requests retain queued semantics and receive an
209+
explicit current-execution refusal. Only the chosen target is consumed.
210+
211+
`current_execution_point()` returns an owning presentation of the active
212+
callback's quote and calendar-derived decision context. AsPresented uses that
213+
price; NearestTick uses the existing rounding primitive. Configured directional
214+
slippage applies once. An ordinary execution notification anchors at its
215+
resolved execution price. A current execution notification inherits its cause's
216+
quote, so a chain does not compound slippage. Finite nonpositive current prices
217+
are permitted for pure closes; opening effects and ordinary matching keep their
218+
positive-price restriction. Native opening admission still decides separately.
219+
220+
`NativeCurrentExecutionPreview::settlement_readiness` reports the financial
221+
pre-source preparation boundary. It is independent of the account projection's
222+
status and excludes later counter/lifecycle checks and opening admission.
223+
Refusals leave readiness absent. Invalid/NoEffect readiness skips source
224+
preflight. Applied readiness requires source preflight even for opening-only
225+
commands whose account projection overflows. Ordered `closed_row_pnl` values
226+
exist only for Applied readiness and use execution's pinned ticket allocation.
227+
Execution revalidates; editing the preview cannot authorize or alter a fill.
228+
229+
The default-empty `on_native_applied(event, context)` notification follows the
230+
Account record and group/owner/dependency drains. Synchronous commands enqueue
231+
notifications FIFO until the outer callback returns and passes projection and
232+
abort checks. Callback depth stays one; event values remain valid during later
233+
submissions. Notifications never automatically execute newborn requests and
234+
have no semantic 64-execution cap. Failure after a physical commit discards the
235+
host; a failed host cannot retry. Configuration projection is checked before
236+
in-callback execution as well as after callback return.
237+
238+
This native slice does not switch generated Pine code to the native consumer
239+
or complete source scheduling, ownership transfer, or parity acceptance.
240+
168241
A `quantity_grid`, when present, admits Transact/Reduce quantities on the
169242
exact binary64 grid in `native_order.hpp`. Flatten is not gridded. Rejection
170243
does not rewrite the attempted bits.
@@ -176,7 +249,7 @@ does not rewrite the attempted bits.
176249
not change.
177250
- `Rejected` — rejection ordinal, no handle (`InvalidQuantity` / `OffGrid`).
178251

179-
Fills appear later as `ExecutionAppliedEvent` on the same command history
252+
Ordinary queued fills appear later as `ExecutionAppliedEvent` on the same command history
180253
(`native_events(0)`). Default market requests retain acceptance/incarnation
181254
priority at an eligible matching driver point. Their birth eligibility is
182255
`point_ordinal > birth.acceptance_ordinal` and

‎include/pineforge/compat/pine/exit_activation.hpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99

1010
namespace pineforge {
1111
enum class PositionSide;
12-
inline namespace engine_script_run_v13 { struct PendingOrder; }
12+
inline namespace engine_script_run_v14 { struct PendingOrder; }
1313
}
1414
namespace pineforge::compat::pine {
1515

‎include/pineforge/compat/pine/exit_lifecycle.hpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#pragma once
22
#include "../../exit_leg_lifecycle.hpp"
3-
namespace pineforge { inline namespace engine_script_run_v13 { struct PendingOrder; } }
3+
namespace pineforge { inline namespace engine_script_run_v14 { struct PendingOrder; } }
44
namespace pineforge::compat::pine {
55
struct ExitSuspensionContext {
66
exit_legs::Frame cause;

‎include/pineforge/compat/pine/frozen_market_instruction.hpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
#include <vector>
1010

1111
namespace pineforge {
12-
inline namespace engine_script_run_v13 { struct PendingOrder; }
12+
inline namespace engine_script_run_v14 { struct PendingOrder; }
1313
}
1414

1515
namespace pineforge::compat::pine {

‎include/pineforge/compat/pine/order_birth.hpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#pragma once
22
#include "../../order_birth.hpp"
33

4-
namespace pineforge { inline namespace engine_script_run_v13 { struct PendingOrder; } }
4+
namespace pineforge { inline namespace engine_script_run_v14 { struct PendingOrder; } }
55
namespace pineforge::compat::pine {
66

77
// Historical Pine permissions remain policy, not physical birth facts.

‎include/pineforge/compat/pine/order_priority.hpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
#include <string>
66
#include <vector>
77

8-
namespace pineforge { inline namespace engine_script_run_v13 { struct PendingOrder; } }
8+
namespace pineforge { inline namespace engine_script_run_v14 { struct PendingOrder; } }
99
namespace pineforge::compat::pine {
1010

1111
struct OrderPriorityContext {

‎include/pineforge/compat/pine/reservation_expansion.hpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
#include <vector>
55
namespace pineforge {
66
enum class PositionSide;
7-
inline namespace engine_script_run_v13 { struct PendingOrder; }
7+
inline namespace engine_script_run_v14 { struct PendingOrder; }
88
}
99
namespace pineforge::compat::pine {
1010
// Called only for omitted explicit exit qty. Pine owns the population selector;

‎include/pineforge/engine.hpp‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -432,7 +432,7 @@ enum class ShortSeedCollisionRole : uint8_t {
432432
// PendingOrder crosses out-of-line helper boundaries independently of the
433433
// engine class, so its changed layout must carry the same internal epoch.
434434
using ExitLegLifecycle = exit_legs::Lifecycle;
435-
inline namespace engine_script_run_v13 {
435+
inline namespace engine_script_run_v14 {
436436
struct PendingOrder {
437437
std::string id;
438438
std::string from_entry; // for exit orders
@@ -960,7 +960,7 @@ inline bool placement_has_opposite_market_predecessor(
960960
return false;
961961
}
962962

963-
} // inline namespace engine_script_run_v13 (PendingOrder)
963+
} // inline namespace engine_script_run_v14 (PendingOrder)
964964

965965
// default_qty_type constants (matches TradingView)
966966
enum class QtyType { FIXED = 0, PERCENT_OF_EQUITY = 1, CASH = 2 };
@@ -1018,7 +1018,7 @@ struct StrategyOverrides {
10181018
// v6 adds explicit owner-bound exit-leg activation and Pine placement evidence.
10191019
// Version the mangled class name so older headers' member offsets/vtable cannot
10201020
// silently bind out-of-line members of this different object layout.
1021-
inline namespace engine_script_run_v13 {
1021+
inline namespace engine_script_run_v14 {
10221022
class BacktestEngine {
10231023
protected:
10241024
friend class LegacyCompatibilityConsumer;
@@ -3912,6 +3912,13 @@ class BacktestEngine {
39123912
const execution::PhysicalExecutionContext& context,
39133913
execution::CloseScope book_or_opening,
39143914
const execution::SelectedOpeningSet* selected);
3915+
// Runtime-private preview of the exact existing pre-source prefix. The
3916+
// caller pins the quoted ticket before entering this method.
3917+
execution::Status preview_native_settlement_commit(
3918+
const execution::Action& action, const execution::Fill& fill,
3919+
const execution::PhysicalExecutionContext& context,
3920+
execution::CloseScope scope, const execution::SelectedOpeningSet* selected,
3921+
execution::AccountEffectProjection& account, std::vector<double>& row_pnl) const;
39153922
struct NativeSettlementStage;
39163923
struct NativeSettlementRows;
39173924
execution::Status validate_native_settlement_book(double& held) const;
@@ -5371,5 +5378,5 @@ class BacktestEngine {
53715378
void trace(const std::string& name, int value) { trace(name, static_cast<double>(value)); }
53725379
};
53735380

5374-
} // inline namespace engine_script_run_v13
5381+
} // inline namespace engine_script_run_v14
53755382
} // namespace pineforge

0 commit comments

Comments
 (0)