Skip to content

Commit 480fd19

Browse files
luisleo526claude
andauthored
Production-readiness probes + engine state-reset / finite-fill fixes (#28)
* docs: production-readiness probe plan, findings, and TV-parity spec Design spec (gap analysis -> ranked plan), source-verified findings doc (fixed F1/F3/F5 + open O2/O4/O5/O6/O7 + validation-methodology disclosures), and a TV-parity probe handoff spec (Families A-E with clean-room .pine sketches + capture recipes). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(engine): reset full per-run state on rerun; reject non-finite fill sizing Two source-verified latent bugs (no corpus impact — both paths are unreachable on the clean, fresh-handle corpus; re-verified 233 excellent + 1 anomaly, 0 regressions): 1. Handle reuse leaked state. run() only reserved trades_; it never cleared trades_/net_profit_sum_/gross sums/counters, open position, pending orders, risk latches, intraday/day counters, or source-series history. The C ABI advertises reuse-then-rerun (sweeps), so run 2 on a reused handle accumulated — could even start already in a position or risk-halted. New reset_run_state() resets the full per-run surface (configuration preserved) at the top of every run loop. 2. Silent wrong-qty fallback. calc_qty/calc_qty_for_type returned the raw percent/cash number as a contract count when fill_price<=0 or NaN (a $0/NaN print sized '10% of equity' as 10 contracts). Now reject (qty 0). execute_market_entry drops a fill at a non-finite price so a bad tick cannot open a position at a NaN/Inf entry. calc_qty_for_type moved to protected so the guard test can exercise it directly. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(probes): engine-only production-readiness probe suite Eight new ctests (+~1450 assertions; 45 -> 53 ctest) pinning claims a skeptic would challenge, all green, no TradingView needed: - determinism_reproducibility : bit-identical fresh-handle reruns - handle_reuse_reset : reused handle == fresh (guards the fix) - exit_path_segment_tiebreak : same-bar TP+SL / gap priority is deterministic (analytic OHLC oracle) - accounting_reconciliation : gross+gross==net, counts, independent per-trade PnL recompute - ta_volume_state_oracle : MFI/CMO/WPR bounds + OBV/PVT/AccDist forward-sum identities - market_structure_fills : mintick FLOOR snap at ES/gold/FX ticks - calendar_aggregation_wm : W/M aggregation (leap Feb, year rollover) - adversarial_ohlcv : NaN/Inf/zero/empty feeds stay finite, no silent qty fallback (guards the fix) Plus scripts/verify_self_test.py: the validator-of-the-validator — proves verify_corpus.py fails on gross-corrupted trade lists, and discloses its sub-1% / trailing-block-trim tolerances honestly. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * feat(validator): report-only field coverage (qty/pnl_pct/MFE/MAE, p100, unmatched) verify_corpus.py historically compared 5 of 10 TradeC fields and gated at p90. It now ALSO parses + reports qty, pnl_pct, MFE, MAE deltas, plus p100 worst-case and an unmatched-in-window count — report-only, the tier gate is unchanged (re-verified 233 excellent + 1 anomaly). Immediately surfaces previously-hidden divergence (e.g. MAE p90 ~200%, a pnl_pct tail) feeding findings O5/O7. Also corrects the pnl_pct ABI doc to match the actual gross price-return formula (was: 'net % of entry capital'). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * ci: AddressSanitizer + UBSan lane PINEFORGE_ENABLE_SANITIZERS existed but CI never set it, so the leak / NaN / double-free probes ran un-instrumented. Adds a dedicated ubuntu ASan+UBSan job (library built with sanitizers PUBLIC so instrumentation propagates into every test binary) running ctest under detect_leaks=1:halt_on_error=1. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(engine): complete reset_run_state fanout (input.source/trace/session) Automated commit security review flagged reset_run_state() as an incomplete state-reset fanout — a reused handle still leaked: - input.source ring buffers _src_ohlc4_ / _src_hlcc4_ (the two newest canonical series were missed) - trace buffers trace_buffer_ / trace_names_ / trace_name_index_ - session-predicate state prev_in_session_ / session_ismarket_ / session_isfirstbar_ / session_islastbar_ All now reset (trace_enabled_ preserved — it is configuration). Comment added requiring EVERY _src_*_ member be listed to prevent the same miss recurring. ctest 53/53; corpus 233 excellent + 1 anomaly, 0 regressions. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * chore: drop internal brainstorming design spec from public repo docs/superpowers/ holds maintainer-only process artifacts (not referenced by README, the docs site, or CI). The production-readiness design spec I added there is internal; its reader-facing content lives in docs/production-readiness-findings.md + docs/tv-parity-probe-spec.md. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * chore: keep internal process docs out of public history Untrack docs/superpowers/ (11 brainstorming plans/specs) and docs/production-readiness-findings.md and add both to .gitignore, matching the existing HANDOFF.md / BENCH_REFRESH_HANDOFF.md policy — these are maintainer-only artifacts not referenced by the README, the docs site (Doxyfile ships only docs/pages/* + the public header), or CI. Files are kept on disk locally, removed from tracking only. Scrub the now-internal findings.md references from the public surface (pineforge.h pnl_pct doc, verify_corpus.py comment, tv-parity-probe-spec) so those stay self-contained. No behaviour change; ctest 53/53. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(ci): consistent sanitizer instrumentation across TUs (Eigen bad-free) The ASan lane failed on Linux: test_matrix aborted with AddressSanitizer 'attempting free on address which was not malloc()-ed' at the PineMatrix destructor. Root cause: sanitizer COMPILE flags were PRIVATE to the library, so a consumer TU (the test) linked the instrumented lib but compiled without -fsanitize -> a different Eigen alignment view than matrix.cpp. An Eigen matrix allocated in the library and freed (header-inlined) in the test then mis-freed the aligned offset pointer. macOS's allocator masked it; Linux caught it. Fix: make -fsanitize compile options PUBLIC so lib + tests + strategy .so are instrumented identically, and define EIGEN_MAX_ALIGN_BYTES=0 under the sanitizer build so Eigen uses plain malloc/free (no offset pointers). Sanitizer-lane only; normal builds keep full alignment. Local ASan/UBSan: 53/53 pass incl. test_matrix. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
1 parent a0535f0 commit 480fd19

30 files changed

Lines changed: 1645 additions & 5657 deletions

‎.github/workflows/ci.yml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,3 +66,31 @@ jobs:
6666
cmake --build smoke_build -j 4
6767
expected="$(tr -d '[:space:]' < VERSION)"
6868
./smoke_build/smoke_version | grep -qxF "${expected}"
69+
70+
# Dedicated AddressSanitizer + UndefinedBehaviorSanitizer lane. Without this,
71+
# the determinism / NaN-propagation / handle-reuse / C-ABI-lifecycle probes
72+
# would run un-instrumented and could not actually trip the leak / double-free
73+
# / UB they target. The library is built with the sanitizers PUBLIC so the
74+
# instrumentation propagates into every test binary (CMakeLists.txt option
75+
# PINEFORGE_ENABLE_SANITIZERS).
76+
sanitizers:
77+
runs-on: ubuntu-latest
78+
steps:
79+
- uses: actions/checkout@v4
80+
81+
- name: Install Eigen3
82+
run: |
83+
sudo apt-get update
84+
sudo apt-get install -y libeigen3-dev
85+
86+
- name: Configure (ASan + UBSan)
87+
run: cmake -B build-asan -DCMAKE_BUILD_TYPE=Debug -DPINEFORGE_ENABLE_SANITIZERS=ON
88+
89+
- name: Build
90+
run: cmake --build build-asan -j 4
91+
92+
- name: Test under sanitizers
93+
env:
94+
ASAN_OPTIONS: detect_leaks=1:halt_on_error=1:abort_on_error=1
95+
UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1
96+
run: ctest --test-dir build-asan --output-on-failure

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,3 +62,7 @@ tutorial/**/trades.csv
6262
HANDOFF.md
6363
BENCH_REFRESH_HANDOFF.md
6464
.claude/
65+
66+
# Internal process artifacts + working notes (kept local, not public history)
67+
docs/superpowers/
68+
docs/production-readiness-findings.md

‎CMakeLists.txt‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -170,10 +170,20 @@ endif()
170170
# ASan + UBSan. Link options are PUBLIC so anything linking libpineforge.a
171171
# (tests, corpus strategy.so) picks up the runtime.
172172
if(PINEFORGE_ENABLE_SANITIZERS)
173-
target_compile_options(pineforge PRIVATE
173+
# Compile options must be PUBLIC, not PRIVATE: a consumer TU (test, strategy
174+
# .so) that links the instrumented library but compiles WITHOUT -fsanitize
175+
# gets a different Eigen alignment view than matrix.cpp, so an Eigen matrix
176+
# allocated in the library and freed (header-inlined) in the consumer hits a
177+
# cross-TU aligned-free mismatch -> ASan "bad-free" (seen on Linux; masked by
178+
# macOS's allocator). PUBLIC keeps instrumentation consistent everywhere.
179+
target_compile_options(pineforge PUBLIC
174180
-fsanitize=address,undefined -fno-omit-frame-pointer)
175181
target_link_options(pineforge PUBLIC -fsanitize=address,undefined)
176-
message(STATUS "Sanitizers: -fsanitize=address,undefined")
182+
# Belt-and-suspenders: disable Eigen's manual over-alignment under the
183+
# sanitizer build so Eigen uses plain malloc/free (no offset pointers to
184+
# mis-free). Sanitizer-lane only; normal builds keep full alignment.
185+
target_compile_definitions(pineforge PUBLIC EIGEN_MAX_ALIGN_BYTES=0)
186+
message(STATUS "Sanitizers: -fsanitize=address,undefined (Eigen alignment disabled)")
177187
endif()
178188

179189
# === Strict warnings (opt-in) =========================================

0 commit comments

Comments
 (0)