Skip to content

Commit 04b0588

Browse files
committed
Add native resolved execution settlement and preserve paid entry costs
1 parent 83c2a1c commit 04b0588

13 files changed

Lines changed: 1285 additions & 85 deletions

‎CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ add_library(pineforge STATIC
8787
src/engine_lower_tf.cpp
8888
src/engine_metrics.cpp
8989
src/engine_orders.cpp
90+
src/engine_execution.cpp
9091
src/engine_path_resolve.cpp
9192
src/engine_report.cpp
9293
src/engine_risk.cpp

‎docs/native-settlement.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Native resolved execution settlement
2+
3+
`BacktestEngine::settle_resolved_execution` is a protected C++ extension for
4+
trusted matching adapters. It applies an already admitted, matched execution
5+
to the engine's existing physical lot book and emits the resulting close and
6+
open observations. It adds no second position ledger or persistent selector.
7+
8+
The caller supplies an `execution::Action` and `execution::Fill` from
9+
`<pineforge/execution.hpp>`. These values describe immediate effects:
10+
11+
| Action | Effect |
12+
| --- | --- |
13+
| `Flatten{}` | Close every physical lot. |
14+
| `order_action::Reduce{units}` | Close up to a finite, nonnegative quantity in FIFO order; never open or reverse. |
15+
| `order_action::Transact{signed_units}` | Buy positive units or sell negative units; reduce opposite exposure first, then open any remainder. |
16+
17+
`Fill` carries the resolved finite price, order ID, comment, incarnation and
18+
an optional total commission in account currency. Zero and negative prices
19+
are representable; the caller decides which prices its instrument permits.
20+
Settlement does not apply another tick rounding, slippage, lot step,
21+
pyramiding cap or affordability decision.
22+
23+
## Physical lots and quantities
24+
25+
FIFO allocation retains surviving lot identities, entry prices, paid costs
26+
and proportional excursions. The surviving physical quantities determine the
27+
resulting position and weighted entry price. For example, reducing lots
28+
`{0.1, 0.2}` by `0.1` leaves the existing `0.2` lot, even when regrouping
29+
binary64 arithmetic as `aggregate_before - reduction` differs by one ulp.
30+
31+
A finite quantity request can leave a positive floating residual. No native
32+
epsilon silently discards it. `Flatten` explicitly closes all lots and avoids
33+
using rounded aggregate equality to mean a whole-book close. Quantities whose
34+
changes cannot be represented are refused before settlement.
35+
36+
## Costs and marked equity
37+
38+
Every opening lot receives its paid entry commission before its entry
39+
observation is emitted. Partial closes allocate this stored payment in
40+
proportion to closed quantity for every commission type. The survivor keeps
41+
the unconsumed cost. Later fee schedule or FX changes do not rewrite a payment
42+
already made.
43+
44+
The current execution is quoted separately. Percent and per-contract charges
45+
follow each physical quantity. A cash-per-order charge belongs to the entire
46+
native execution, including both legs of a reversal; it is allocated across
47+
the close/open effects with the final effect receiving the floating residue.
48+
An explicit `Fill::commission_account` overrides the modeled total and can
49+
represent a zero-fee waiver or a negative rebate. A nonfinite quote, or a
50+
nonzero quote attached to a no-effect quantity, is refused.
51+
52+
For example, with a cash ticket of 6, opening 3 units and reducing 1 unit
53+
allocates 2 of the paid entry ticket plus 6 for the reduction to the closed
54+
row. The remaining 2 units carry entry cost 4. Flattening those units costs
55+
another ticket of 6: total paid across the three executions is 18.
56+
57+
`marked_equity(price)` is initial capital plus realized PnL and marked open
58+
PnL, minus remaining paid entry costs of every fee type. At an unchanged mark,
59+
a native execution decreases this value by its current charge, subject to
60+
ordinary floating-point rounding. A rebate increases it.
61+
62+
## Integration and limits
63+
64+
The kernel currently serves full market exits and selected frozen-transaction
65+
and same-side materialization paths. Existing source scheduling and dust
66+
decisions remain at their call sites. Some source instructions still issue
67+
separate close and open executions. Other legacy close loops are not yet
68+
grouped into one parent execution; their per-row current ticket behavior is
69+
not a claim about the native contract.
70+
71+
The shared close builder now consumes historical entry costs. This changes
72+
the former reconstruction that converted both commission legs at exit-time
73+
FX, and the former reuse of a complete entry ticket at every partial close.
74+
TradingView compatibility must be measured against unchanged references.
75+
Pine sizing and range-end reporting still have separate conventions; the
76+
native marked-equity helper does not replace all existing report fields.
77+
78+
This API is synchronous. It does not place queued orders, establish matching
79+
or cancellation authority, reconcile external broker messages, deduplicate
80+
replayed fills or accept a stored plan as execution authority. Callers own
81+
those responsibilities and the current event context. The return status
82+
distinguishes application, no effect and validation failures. Allocation or
83+
lifecycle exceptions abort the owning run; callers must discard that failed
84+
run rather than retry a partially committed execution in place. Strong
85+
rollback on allocation failure is not promised.
86+
87+
The work does not remove `ShortSeedCollisionRole` or complete migration to a
88+
generic queued order machine. Existing executable state remains represented
89+
in ABI projections and fingerprints.

‎docs/pages/fill-model.md‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -104,14 +104,15 @@ authorize a delayed fill: a scheduler must revalidate its order identity and
104104
current book before settlement. Pending cancellation and replay receipts are
105105
separate lifecycle contracts.
106106

107-
The current bridge uses this planner for partial reduction and Pine's already
108-
resolved frozen-transaction close/open split. The existing settlement paths
109-
still own FIFO lots, fees, slippage, dust thresholds and observations. A shared
110-
same-side lot append also keeps physical identity and metadata without
111-
introducing another position book. This is a prerequisite for the unified
112-
executor; `ShortSeedCollisionRole` and its compatibility projection remain
113-
until ordered actions replace all of their consumers. This change does not
114-
reduce the pending-order flag count or provide broker-account reconciliation.
107+
The native [resolved settlement extension](../native-settlement.md) applies
108+
`Flatten`, `Reduce` and `Transact` to the existing physical FIFO book, including
109+
paid entry costs, current execution charges and ordered observations. Full
110+
market exits and selected frozen-transaction/materialization paths use it.
111+
Callers still own matching, admission, slippage and source scheduling; other
112+
legacy settlement paths remain to be migrated. `ShortSeedCollisionRole` and
113+
its compatibility projection remain until ordered actions replace all of
114+
their consumers. This step does not reduce the pending-order flag count or
115+
provide broker-account reconciliation.
115116

116117
## Opening checkpoint
117118

‎docs/pages/metrics.md‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,19 @@ qty × pointvalue) × 100`, TradingView-arbitrated).
4949
| `max_consecutive_wins` / `_losses` | Longest run; even trades reset both | — |
5050
| `avg_bars_in_trade` / `_wins` / `_losses` | Mean of `exit_bar − entry_bar + 1` (**inclusive** of the entry bar, TV convention), script bars | empty set |
5151

52+
Native settlement records fees in account currency at the event where they are
53+
incurred. Entry fees are snapshotted on each physical lot at its entry fill
54+
and remain unchanged when a later FX point becomes active. A partial FIFO
55+
close allocates the paid entry fee in proportion to closed quantity; the
56+
survivor retains the unconsumed cost. A
57+
single cash-per-order ticket belongs to the native execution as a whole and is
58+
allocated proportionally across its physical close/open effects; it is not
59+
charged once per trade row. The shared native close helper now consumes these
60+
historical paid costs. Existing legacy Pine paths are not all grouped into
61+
native executions, so this rule does not claim that every legacy path has the
62+
same dynamic-FX settlement semantics. Pine sizing, source-specific reporting,
63+
and range-end conventions remain compatibility behavior.
64+
5265
## Equity statistics (`metrics.equity`, all-trades only)
5366

5467
The equity curve samples `initial_capital + net_profit + open_profit` at
@@ -77,7 +90,7 @@ truncated curve and metrics over the truncated prefix.
7790
| Surface | Validated against | Result |
7891
| --- | --- | --- |
7992
| Trade statistics (counts, PF, percent bases, averages, largest-%, bars) | Real TradingView Strategy Tester export (`composite-4emarsi-integration-01`, 336 trades, All/Long/Short panels) | Match within TV 2-dp rounding; three TV conventions arbitrated and adopted (net return-on-cost `pnl_pct`, independent largest-%, inclusive bar counts) |
80-
| Commission + slippage economics | Second TV export, same strategy: commission 0.1 % percent + slippage 2 ticks via `strategy_set_override` ([historical inputs](https://github.com/pineforge-4pass/pineforge-engine/blob/a03ac6d3fb42df5af1db9e39727daf450b2fb71f/validation-adhoc/4emarsi-commission-slippage-ethusdt/inputs.json)) | All 672 fill prices bit-exact (slippage rules pinned: market/stop fills, directional, mintick-composed); commission formula `rate·(entry+exit)·qty·pointvalue` exact — residual per-trade deltas fully explained by TV's account-currency conversion (USDT→USD at previous-UTC-day close; 335/336 trades reproduced to the cent). Extended by a third TV export (`bracket-exit-tp-sl-fixed-01`, BINANCE:ETHUSDT.P): 396/396 trades bit-exact, pinning the limit-fill rules — limit fills are NOT slipped, off-tick limits snap one tick favorably (limit-or-better), gapped limits fill at the raw open, and stop fills confirmed slipped |
93+
| Commission + slippage economics | Historical TradingView exports: 0.1% commission + 2 ticks via `strategy_set_override` ([archived inputs](https://github.com/pineforge-4pass/pineforge-engine/blob/a03ac6d3fb42df5af1db9e39727daf450b2fb71f/validation-adhoc/4emarsi-commission-slippage-ethusdt/inputs.json)) | Historical fixed-export validation only: 672 fill prices bit-exact and 335/336 commission rows reproduced to the cent after the documented USDT→USD conversion. A separate historical bracket export reproduced 396/396 trades and pinned limit/stop slippage rules. These archived receipts do not claim current dynamic-FX native settlement parity; current native settlement preserves paid entry costs and allocates one execution's costs across physical effects. |
8194
| TV risk panel (Sharpe, Sortino, drawdown/run-up rows, CAGR) | TV xlsx export (Performance + Risk-adjusted performance sheets) | Every panel value reproduced from the engine curve once TV's conventions are applied — see the definition-delta table below |
8295
| Equity statistics (max DD ±%, Sharpe/Sortino both variants, CAGR, Calmar, recovery) | quantstats 0.0.81 + empyrical-reloaded 0.5.12 (`scripts/crossvalidate_metrics.py --all`) | All 246 corpus strategies ran, 0 skipped, 0 mismatches; worst engine-convention \|rel Δ\| = 1.886e-11 (`pyramid-cash-fractional-commission-01`, sharpe/sortino_bar vs empyrical); 3 degenerate NaN fields (sharpe_tv, zero monthly variance) agree on degeneracy across engine/numpy/empyrical/quantstats; known library-convention deltas labelled in single-strategy mode |
8396
| Closed-form unit oracles | `tests/test_metrics.cpp` (e.g. monthly Sharpe 19/20, Sortino 114/61 exact rationals) | Bit-level |

‎include/pineforge/engine.hpp‎

Lines changed: 36 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
#include "bar.hpp"
1717
#include "broker_events.hpp"
1818
#include "quantity_intent.hpp"
19+
#include "execution.hpp"
1920
#include "market_admission.hpp"
2021
#include "reservation_expansion.hpp"
2122
#include "order_cancellation.hpp"
@@ -248,10 +249,10 @@ struct PyramidEntry {
248249
// Entry-leg commission in account currency at this slice's actual fill
249250
// boundary. Percentage commission depends on quote->account FX, so an
250251
// effective-time provider must not retroactively reprice this already-paid
251-
// fee when a later rate becomes active. Closed-trade reporting deliberately
252-
// ignores this snapshot: TV converts the complete realized trade at the
253-
// exit-time FX rate. NaN is reserved for legacy/synthetic test injection;
254-
// every production entry path initializes the snapshot at fill time.
252+
// fee when a later rate becomes active. Partial realization allocates this
253+
// paid cost proportionally and leaves only the unconsumed cost on the lot,
254+
// independently of the current fee schedule. NaN is reserved for legacy/
255+
// synthetic injection; every production entry path captures a real quote.
255256
double entry_commission_account =
256257
std::numeric_limits<double>::quiet_NaN();
257258
// Monotonic per-run identity of the PendingOrder object whose broker fill
@@ -1790,6 +1791,18 @@ class BacktestEngine {
17901791
// --- Per-trade extreme tracking ---
17911792
void update_per_trade_extremes();
17921793

1794+
// Trusted native matching-adapter extension: settle an already resolved
1795+
// execution into this engine's one physical book. This does not place an
1796+
// order, perform admission/slippage, or provide cancellation/replay. The
1797+
// owning run must abort on an exception; failed commits are not retryable
1798+
// in place. A saved arithmetic plan is not execution authority.
1799+
execution::Result settle_resolved_execution(const execution::Action& action,
1800+
const execution::Fill& fill);
1801+
// Native account value: realized balance plus marked physical lots minus
1802+
// their remaining paid entry costs, for every fee type. No Pine sizing or
1803+
// end-of-range reporting convention participates in this value.
1804+
double marked_equity(double price) const;
1805+
17931806
// --- Strategy order commands ---
17941807
// NOTE: prior to v0.2 the runtime accepted a leading `double market_price`
17951808
// positional after `is_long`. The implementation never read it; every
@@ -2428,6 +2441,12 @@ class BacktestEngine {
24282441
: calc_commission(pe.price, pe.qty);
24292442
}
24302443

2444+
double allocated_entry_commission(const PyramidEntry& pe, double units) const {
2445+
if (units <= 0.0) return 0.0;
2446+
const double paid = open_entry_commission(pe);
2447+
return units >= pe.qty ? paid : paid * (units / pe.qty);
2448+
}
2449+
24312450
void snapshot_entry_commission(PyramidEntry& pe) const {
24322451
pe.entry_commission_account = calc_commission(pe.price, pe.qty);
24332452
}
@@ -3851,6 +3870,8 @@ class BacktestEngine {
38513870
uint64_t entry_incarnation);
38523871
void execute_market_exit(double fill_price);
38533872
void append_same_side_fill(PyramidEntry lot);
3873+
void append_quoted_lot(PyramidEntry lot, double total_qty, double average_price);
3874+
void open_quoted_position(PositionSide requested, PyramidEntry lot);
38543875
// Range-end accounting: record the rows that close a position still
38553876
// open after the final script bar at that bar's close, the way
38563877
// TradingView's deep-backtest report does (engine_orders.cpp). Called by
@@ -4200,13 +4221,24 @@ class BacktestEngine {
42004221
// engine_orders.cpp).
42014222
void emit_close_trade(const PyramidEntry& pe, double close_qty,
42024223
double fill_price, bool was_long);
4224+
void record_close_trade(Trade trade);
4225+
void validate_close_trade_counters(const Trade* rows, size_t count) const;
4226+
// Quote one resolved execution's current charges. Entry costs on the
4227+
// closed rows are historical allocations. Returns close shares in FIFO
4228+
// order followed by the opening share (zero when there is no opening).
4229+
std::vector<double> quote_execution_commissions(
4230+
const std::vector<double>& closed_units, double opening_units,
4231+
const execution::Fill& fill) const;
42034232
// The arithmetic of emit_close_trade without its bookkeeping: the Trade
42044233
// row a close of ``close_qty`` of ``pe`` at ``fill_price`` on the
42054234
// current bar would record (pnl, pnl_pct, commission, excursions, bar
42064235
// indexes). emit_close_trade builds and commits; the range-end close
42074236
// builds only.
42084237
Trade build_close_trade(const PyramidEntry& pe, double close_qty,
42094238
double fill_price, bool was_long) const;
4239+
Trade build_close_trade_with_costs(const PyramidEntry& pe, double close_qty,
4240+
double fill_price, bool was_long, double entry_commission,
4241+
double exit_commission) const;
42104242
// FIFO-drain up to qty_limit from pyramid_entries_, in order, splitting the
42114243
// boundary entry as needed. When from_entry is non-null only entries whose
42124244
// entry_id == *from_entry are eligible (others are kept untouched); null

‎include/pineforge/execution.hpp‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#pragma once
2+
3+
#include "order_action.hpp"
4+
#include <cstdint>
5+
#include <optional>
6+
#include <string>
7+
#include <variant>
8+
9+
namespace pineforge::execution {
10+
11+
// Whole-book reduction is explicit: it never relies on a rounded aggregate
12+
// quantity being exactly equal to the sum of individual physical lots.
13+
struct Flatten {};
14+
using Action = std::variant<Flatten, order_action::Reduce, order_action::Transact>;
15+
16+
// The caller has already matched/admitted the action and resolved its price.
17+
// Settlement does not apply another lot step, price grid, slippage or entry cap.
18+
struct Fill {
19+
double price;
20+
std::string id;
21+
std::string comment;
22+
uint64_t incarnation = 0;
23+
// An observed execution charge in account currency, including rebates.
24+
// Otherwise the engine's configured schedule quotes the current fill.
25+
std::optional<double> commission_account = std::nullopt;
26+
};
27+
28+
enum class Status {
29+
Applied, NoEffect, InvalidPrice, InvalidQuantity, InvalidBook,
30+
UnrepresentableQuantity, InvalidAccounting
31+
};
32+
33+
struct Result {
34+
Status status = Status::NoEffect;
35+
// Aggregate projections of the physical effects; the lot/trade roster is
36+
// authoritative. Opening units are signed; closing units are nonnegative.
37+
double closed_units = 0.0;
38+
double opened_units = 0.0;
39+
};
40+
41+
} // namespace pineforge::execution

0 commit comments

Comments
 (0)