Repository navigation
feat(matrix): typed/UDT matrix support — PineGenericMatrix<T> - #1
Merged
Merged
Conversation
…SOURCES Code review issues from spec compliance pass: - add_col silently read past end of values vector when shorter than rows (UB); add_row had similar silent jagged-row hazard. Both now throw on size mismatch, matching PineMatrix::add_col parity. - Test target registered as one-off bypassed TEST_SOURCES foreach loop used for coverage instrumentation; moved into TEST_SOURCES.
Add explicit PineGenericMatrix<bool> specialization using std::vector<std::vector<char>> internal storage to avoid the vector<bool> proxy footgun. Cast at boundary: bool→char(1/0) on input, char!=0→bool on output. row_ref intentionally absent. Add test_generic_matrix_bool_proxy; 19/19 ctest targets pass.
Series<PineGenericMatrix<UDT>> hit a link-time gap because na.hpp only
specialized double/int/int64_t/bool. Provide a body for the primary
template that returns T{} for any default-constructible T; existing
explicit specializations still take precedence for primitive types.
Removes the local na specialization workaround from the Series test
and adds direct fallback coverage in test_na_runtime.
Move matrix<UDT> from 'Not implemented anywhere' to first-class supported, add PineGenericMatrix<T> module row, and add a 'Typed matrices' coverage row alongside 'Numeric matrices'.
Adds header-only PineGenericMatrix<T> for matrix<int/bool/string/color/UDT> alongside existing PineMatrix (Eigen, double-only). C ABI unchanged (generic matrix is internal compiler-emitted code; never crosses .so boundary). Co-Authored-By: Claude Opus 4.7 <[email protected]>
2 tasks done
luisleo526
added a commit
that referenced
this pull request
May 12, 2026
Pin the contract that strategy.entry's same-id replacement is a bar- boundary event: the prev bar's pending order is evaluated against the new bar's OHLC FIRST, then on_bar runs and any same-id placement takes effect for the NEXT bar (never the current one). Four scenarios: 1. Filled stop unaffected by same-bar same-id replace — verifies process_pending_orders runs before on_bar so the prev-bar stop gets its OHLC evaluation regardless of any modify guard. 2. Unfilled stop replaced for next bar — verifies the new placement is what fires next, not the old one. 3. New entry after same-bar fill defers to next bar — verifies the engine does NOT add a second process_pending_orders pass that would let on_bar's new placements fire on the same bar. 4. Carry capture on flat session start — pins the ``tv_carry_qty=0`` contract for the very first strategy.entry call (regression test for the ``position_qty_=1.0`` default leak that produced qty=2 trade #1 on probe 62).
luisleo526
added a commit
that referenced
this pull request
Sep 23, 2026
…form through one allocator (INT17b) The sanitizers lane of PR #279 aborted two lane PERF-P4 rows, test_native_command_history_read and test_adapter_receipts_in_place: ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new vs free) #1 operator delete(void*, unsigned long) tests/test_native_command_history_read.cpp:66 allocated by: operator new(unsigned long, std::nothrow_t const&) (asan_new_delete.cpp) Each test replaced only the plain/array operator new and the plain/sized operator delete, to count allocations. libstdc++'s temporary buffers (std::inplace_merge in NativeExecutionConsumer::events_after, std::stable_sort in PineExecutionAdapter::apply_open_market_admission) take their memory from the NOTHROW operator new -- left as ASan's own -- and give it back through the SIZED operator delete, which the test had routed to free(). Two allocators in one executable; ASan stops at the mismatch. tests/global_allocation_replacement.hpp is now the one replacement set: all twenty replaceable forms (plain, array, nothrow, sized, aligned and aligned nothrow, for new and for delete) go to one allocator, std::malloc (posix_memalign past the fundamental alignment) released by std::free, with the standard new-handler loop and nothrow semantics. A test reads and sets plain state: `allocations` (every allocation, measured as a difference around a span), `refusing` / `refused` (every form fails the way it fails, nothing allocated), and, opted in with PINEFORGE_TEST_ALLOCATION_BYTES, per-block size headers for a recording window (`recording`, `live_bytes`, `on_recorded`). Every test that replaced the global functions now includes it instead of its own partial set -- all of them were incomplete (no aligned forms; most had no nothrow forms either), so a libstdc++ path through the missing forms would have mismatched the same way: test_native_command_history_read P4 counted, was plain+sized only test_adapter_receipts_in_place P4 counted, was plain+sized only test_native_calendar_memo_cost K1 counted, no aligned forms test_native_batch_log_presize K24 byte window, no aligned forms test_native_order_resting_core counted, partial aligned/nothrow test_native_selected_core counted, plain+sized only test_l10ae_allowance_refresh_liveness counted, plain+sized only test_l8b_pending_mirror refused, no aligned forms test_market_admission_causality refused, plain+sized only test_native_run_spec refused, plain(+sized) only test_pending_order_mirror_no_alloc_l4d refused, plain+sized only; its base body keeps its two names, now references to the shared state The uncompiled legacy base tests/test_pending_order_mirror_no_alloc.cpp (REMOVE_ITEM'd from TEST_SOURCES; its ab9714b body is the twin's parity reference) is left as it is. No assertion, bound or pinned value changes: the counted windows read the same numbers (calendar memo cost: canonical 159, tolerant 88, 0.02 extra per bar; batch presize: 22522624 held for 22176000 recorded, 1.016x; the P4 rows 57356 and 98 checks), and nothing under src/ or include/ changes. Proof, the sanitizers profile reproduced on spark (Linux aarch64, GCC 13.3.0-6ubuntu2~24.04.1): ci_verify's own configure argv and ctest flags, the six ABI providers prepared, the job's ASAN_OPTIONS/UBSAN_OPTIONS. before (f71cd82): both rows abort with the report above, exactly as CI; after, same options: the other ten converted rows pass, including test_adapter_receipts_in_place (96.6 s); the whole suite, with detect_stack_use_after_return=0 added (aarch64 GCC 13's fake stacks cost 30-130x per row on that host: 15.56 s against 0.12 s for one row that CI's x86 runs in 0.51 s): before 99% tests passed, 2 tests failed out of 588 (the two rows) after 100% tests passed out of 588 Mac release, debug and kernel builds: every converted row passes. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Sep 23, 2026
…form through one allocator (INT17b) The sanitizers lane of PR #279 aborted two lane PERF-P4 rows, test_native_command_history_read and test_adapter_receipts_in_place: ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new vs free) #1 operator delete(void*, unsigned long) tests/test_native_command_history_read.cpp:66 allocated by: operator new(unsigned long, std::nothrow_t const&) (asan_new_delete.cpp) Each test replaced only the plain/array operator new and the plain/sized operator delete, to count allocations. libstdc++'s temporary buffers (std::inplace_merge in NativeExecutionConsumer::events_after, std::stable_sort in PineExecutionAdapter::apply_open_market_admission) take their memory from the NOTHROW operator new -- left as ASan's own -- and give it back through the SIZED operator delete, which the test had routed to free(). Two allocators in one executable; ASan stops at the mismatch. tests/global_allocation_replacement.hpp is now the one replacement set: all twenty replaceable forms (plain, array, nothrow, sized, aligned and aligned nothrow, for new and for delete) go to one allocator, std::malloc (posix_memalign past the fundamental alignment) released by std::free, with the standard new-handler loop and nothrow semantics. A test reads and sets plain state: `allocations` (every allocation, measured as a difference around a span), `refusing` / `refused` (every form fails the way it fails, nothing allocated), and, opted in with PINEFORGE_TEST_ALLOCATION_BYTES, per-block size headers for a recording window (`recording`, `live_bytes`, `on_recorded`). Every test that replaced the global functions now includes it instead of its own partial set -- all of them were incomplete (no aligned forms; most had no nothrow forms either), so a libstdc++ path through the missing forms would have mismatched the same way: test_native_command_history_read P4 counted, was plain+sized only test_adapter_receipts_in_place P4 counted, was plain+sized only test_native_calendar_memo_cost K1 counted, no aligned forms test_native_batch_log_presize K24 byte window, no aligned forms test_native_order_resting_core counted, partial aligned/nothrow test_native_selected_core counted, plain+sized only test_l10ae_allowance_refresh_liveness counted, plain+sized only test_l8b_pending_mirror refused, no aligned forms test_market_admission_causality refused, plain+sized only test_native_run_spec refused, plain(+sized) only test_pending_order_mirror_no_alloc_l4d refused, plain+sized only; its base body keeps its two names, now references to the shared state The uncompiled legacy base tests/test_pending_order_mirror_no_alloc.cpp (REMOVE_ITEM'd from TEST_SOURCES; its ab9714b body is the twin's parity reference) is left as it is. No assertion, bound or pinned value changes: the counted windows read the same numbers (calendar memo cost: canonical 159, tolerant 88, 0.02 extra per bar; batch presize: 22522624 held for 22176000 recorded, 1.016x; the P4 rows 57356 and 98 checks), and nothing under src/ or include/ changes. Proof, the sanitizers profile reproduced on spark (Linux aarch64, GCC 13.3.0-6ubuntu2~24.04.1): ci_verify's own configure argv and ctest flags, the six ABI providers prepared, the job's ASAN_OPTIONS/UBSAN_OPTIONS. before (f71cd82): both rows abort with the report above, exactly as CI; after, same options: the other ten converted rows pass, including test_adapter_receipts_in_place (96.6 s); the whole suite, with detect_stack_use_after_return=0 added (aarch64 GCC 13's fake stacks cost 30-130x per row on that host: 15.56 s against 0.12 s for one row that CI's x86 runs in 0.51 s): before 99% tests passed, 2 tests failed out of 588 (the two rows) after 100% tests passed out of 588 Mac release, debug and kernel builds: every converted row passes. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Sep 24, 2026
… new/delete form through INT17b's one allocator
Four slice-B rows defined their own partial set of the replaceable global
allocation functions to count heap allocations -- the pattern INT17b found
aborting under the Linux sanitizers job (alloc-dealloc-mismatch: libstdc++'s
temporary buffers take the NOTHROW operator new, left as ASan's, and give it
back through a SIZED operator delete the test had routed to free()):
test_native_intrabar_scratch PERF-L1 plain/array/nothrow new, plain/sized delete
test_adapter_quiet_bar (+ its PERF-L4 plain/array new, plain/sized delete
_probe and _ungated builds)
test_adapter_command_allocations PERF-L5 plain/array new, plain/sized delete
test_native_match_allocations PERF-L5 plain/array new, plain/sized delete
Each now includes tests/global_allocation_replacement.hpp (all twenty forms,
one malloc/free allocator) instead:
* intrabar_scratch, command_allocations, match_allocations take the
difference of global_allocation::allocations around the span they
counted before (the run; the one re-issued strategy.exit call; the
window between script bars kFirstCounted and kLastCounted), the
style of INT17b's counted conversions; their on/off flag had no other
reader and is gone.
* quiet_bar keeps its body: `allocations` is now a reference to the
shared count (the _l4d twin's precedent), reset where it was reset,
and `count_allocations` still says whether a bar records its count.
Every recorded span lies inside the run it was counted over, so each
per-bar entry is the count it was.
Nothing under src/ or include/ changes (engine code has no over-aligned
allocation, so the aligned forms add nothing to a count). The four rows
print exactly what they printed before the change, line for line, on the
Mac release build: intrabar 0.003-0.045 allocations/bar per path, quiet
bars 4 per stretch (nothing live: 4) magnifier off and on, the re-issued
exit 301 allocations over 42 calls, the matcher 0 per window for every
book. No assertion, bound or pinned value changes.
Fail-before, the sanitizers profile on spark (Linux aarch64, GCC 13.3,
ci_verify's own configure argv, the job's ASAN_OPTIONS/UBSAN_OPTIONS plus
detect_stack_use_after_return=0): the four TUs as the lanes wrote them,
compiled with the sanitizer build's own compile and link commands --
test_adapter_quiet_bar aborts: AddressSanitizer:
test_adapter_command_allocations alloc-dealloc-mismatch (operator new vs free)
#1 operator delete(void*, unsigned long) <the test's own sized delete>
#2 std::__detail::__return_temporary_buffer<PendingSameBarCommand>
#5 std::stable_sort in PineExecutionAdapter::flush_pending_same_bar_commands
allocated by: operator new(unsigned long, std::nothrow_t const&) (asan)
test_native_intrabar_scratch passes (8 passed)
test_native_match_allocations passes (ok)
-- the mechanism INT17b fixed, so both adapter rows would have failed CI's
sanitizers job; the two kernel rows are converted so no counting test is
left with a partial set. After: all four pass there. A grep of tests/ for
global operator new/delete definitions now finds only the helper and the
uncompiled legacy base tests/test_pending_order_mirror_no_alloc.cpp that
INT17b left as it is.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Sep 26, 2026
…blings its leg had (R5 lane INT26, round 2: the gate sweep's leaver) INT26's gate sweep (exp-r5-int26-20260926 against rebaseline e69aa1f7) refused the merge on one leaver, scrapper:data-BINANCE-BTCUSDT/standard/vasudevshenoy-manoj-betrayed-me (BTCUSDT 15m, process_orders_on_close, pyramiding 0; stop entries 'Buy'/'Sell' with strategy.exit stop/limit brackets, a pending entry cancelled when the next bar does not trigger it): 821 of TradingView's 821 trades on main 0d76a09, 820 on ebe6748 (canonical 100 -> 99.9). Bisect (the probe's own feed, spark, one build per pick boundary): a6e6a00 and 2922ade book 821, e1d7a98 -- R5 lane PAR-ORDERS, H-MEASURE Finding 6b, a process_orders_on_close stop entry its placing close already reached fills at that close -- books 820, and so does every later pick. Mechanism, on 2025-08-03 10:00 UTC. The 09:45 close cancels its pending 'Buy', places it again (stop 113689.29) and re-issues 'Buy Exit' (limit 113771.41); the re-issued legs keep their older book slot, so on the 10:00 bar the pre-script drain settles the limit on its parent's touch bar -- after that bar's receipt read. The 10:00 bar closes at its high 113771.47; its body cancels 'Buy' again and places it at that high with a fresh 'Buy Exit' (limit 114004.53), and since e1d7a98 the close pass fills that re-entry at the close, as TradingView does (#274), so the fresh legs are working when 10:15 opens. The first limit's terminal receipt is read only then, and observe_terminal_receipts' sibling cancel withdrew every working leg of the same exit id, from_entry and origin -- the fresh bracket's included, both legs at origin 0. The re-entry ran on to the 15:15 session close and swallowed TradingView's #275 (11:45). Before e1d7a98 the re-entry rested to 10:15, its legs still staged, so the late read found no sibling to take. The pick is right; the sibling cancel was not. Fix, adapter only: cancel_bracket_siblings takes the ordinal the terminal receipt executed at when the observer reads it, and skips a matching leg of a later strategy.exit call (another command_sequence) that the kernel accepted after that ordinal (its Birth::acceptance_ordinal, read through native_working_requests() only when such a leg is among the matches): that leg is the same id's later order. A leg of the filled leg's own call is its sibling however late it was accepted -- the drain submits a bracket's legs one at a time, executing each, so a stop can be accepted after its limit filled -- and goes as before (a review of the first form, which skipped every leg accepted after the fill, found that case). The close passes that call it synchronously pass nothing and are unchanged. No state is added. The probe (spark, generated.cpp of codegen a4259656, the lane feed 6b54c44a): 821 trades, canonical rubric excellent, countAbsDelta 0 (820 / strong before). TradingView's #273-#275 exact: 10:00 113689.29 -> 10:00 113771.41; 10:00 113771.47 -> 11:00 114004.53; 11:45 114005.84 -> 15:15 113844.70. 819 of 821 trades exact to the minute and cent on both legs (main: 771; the two others, #1 and #3, differ on main too). Witness: tests/test_pooc_stop_reentry_bracket_tapes.cpp replays three `lab tv --no-note` tapes of synthetic scripts of the same shape on NYSE:F 15m (tests/fixtures/pooc_stop_reentry_bracket; tv_trades e0447b53 / b7e89ce1 / efbd25e3): three flat signal closes that each cancel and place again a stop entry with its bracket, 10c beyond the close, then 1c (the next bar fills the entry and then the limit and closes flat), then at that close (filled there), long on bars 39-41 and short on bars 44-46; TradingView books the same four rows on the chart and under the magnifier. Every row is compared field by field, with the end book and its working bracket legs. - int26-reentry-bracket-chart: all four rows, flat; the short is the leaver's shape. - int26-reentry-bracket-mag records a divergence the fix does not touch (the same before and after): there the drain leaves the short's retained limit to the flush below the body, whose book still holds the short, so the re-entry is never placed -- rows 1-3 exact, both books flat. - int26-reentry-noexit-chart, the control: the short's re-entry places no strategy.exit (TradingView's row 4 still open at the end); the short bracket's stop, accepted after its limit filled, must still be withdrawn. Fail-before: against ebe6748 2 of 27 checks fail (the chart tape: 3 rows against 4, short 100 against flat); against the first form of the rule 1 of 28 (the control's stop left working); after, 28 of 28. Floors: one source-bound row. RELEASE_MIN_TESTS 702 -> 703, EXCLUDED_REGISTERED_MIN 683/683/692 -> 684/684/693 (and test_ci_verify's expectation, docs/ci.md). KERNEL_MIN_TESTS unchanged. docs/ci.md's sentence "The PR registration floors at INT26 are 683 for Debug and sanitizers and 692 for native" is rewritten with the new counts. Docs: the design page's PAR-ORDERS paragraph records the rule and the magnified divergence; the CHANGELOG 1.0.0 entry's process_orders_on_close item adds it for a user. Every doc anchor carried onto this tree by the exact line map from ebe6748 (135 moved); the three citations of ci_verify.py's RELEASE_MIN_TESTS line (CONTRIBUTING.md, contributing-llm.md twice) set by hand, 390 -> 393. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 1 of typed matrix support: header-only
PineGenericMatrix<T>template alongside existingPineMatrix(Eigen, double-only).Pairs with codegen `feat/typed-matrix-dispatch` (filed separately) which routes `matrix` declarations to this template.
Test plan