Repository navigation
hftbacktest 2.4.4 as the independent sim-lane cross-check engine: pinned install, upstream tests, L1 feasibility - #232
Merged
seathatflowsinourveins merged 8 commits intoSep 25, 2026
Conversation
…im lane Pins hftbacktest==2.4.4 (py-v2.4.4/rust-v0.9.4, MIT) into an isolated venv with a hashed uv lockfile, records upstream's own Python test result at the pinned tag (fails on a missing private fixture; no CI runs it either), and records that no Rust toolchain was available to run cargo test (unresolved, not silently skipped). Builds and hand-verifies a synthetic L1 (Alpaca-SIP- shaped, single-level depth + side-less trades) feasibility fixture against hftbacktest's actual exchange-model source: marketable IOC fill/expiry is decided purely from top-of-book depth (feasible), while trade-driven queue depletion requires an aggressor side that SIP consolidated trades do not carry (feasible-with-limits, demonstrated empirically, not just asserted). Does not touch blueprints/us-equities/sim-capacity/ (parallel builder's work); records the planned cross-check against it once that lane lands on main. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Installs a user-level Rust toolchain (rustup, rustc/cargo 1.98.1) and runs hftbacktest's own #[test] suite at the pinned py-v2.4.4/rust-v0.9.4 tag: `cargo test -p hftbacktest --no-default-features --features backtest --lib` passes 22/22 (0 failed), covering the queue/depth-model unit tests this cross-check's L1 conclusions rely on. The bare default-feature `cargo test` still fails to build (the `live` feature needs libclang + C headers this host lacks and has no sudo to install); recorded as a distinct, narrower, unresolved gap rather than papered over. Re-runs the L1 feasibility fixture (Scenarios A/B/C) under both no_partial_fill_exchange and partial_fill_exchange: results are identical, so the feasible-with-limits verdict is unchanged on this fixture (order sizes never exceeded the touch's quoted quantity, so the two exchange models could not diverge here -- recorded as an open gap, not claimed covered). Adds a Lee-Ready aggressor-side inference step, presented as proposed data preparation (not an hftbacktest feature), and shows it restores trade-driven queue depletion on the same synthetic fixture. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…cker HIGH: .github/osv-scanner-lockfiles.json was missing a covered_by_lockfile entry for the new requirements.in (mover-v3 pattern), which failed test_osv_lockfile_coverage.py. Added and re-verified. MEDIUM, all grounded in a fresh read of the pinned Rust source: - Corrected "8 files, 22 tests cover..." to the actual 7 files run under `-p hftbacktest` (connector/src/utils.rs is a different crate, excluded); explicitly states none of the 22 tests exercise exchange fill logic, IOC/FOK/GTX, ProbQueueModel, or the L2 depth-update path this fixture depends on, and that --lib skips 2 doctests (types.rs, latency.rs). - Moved unrun claims (FOK, GTX, interpolated/per-symbol latency, FlatPerTradeFeeModel) out of "feasible": FOK and GTX are now actually run (Scenario G) and nonzero fees are now actually verified (Scenario H); interpolated/per-symbol latency and FlatPerTradeFeeModel are moved to "not exercised". - Fixed the side-less-fill mechanism: depth alone (on_bid_qty_chg) never invokes is_filled() and never fills a resting order (Scenario D2, new, corrects an earlier wrong claim); passive fills need the OPPOSITE quote to cross (Scenario D3, new); trade-through fills are side-gated too, not just queue depletion (Scenario D1, new). Added the reviewer's three probes as tests. - Renamed the mislabeled "Lee-Ready" rule to the correct EMO at-quote rule (Ellis, Michaely & O'Hara 2000; comparison at NBER w14158) and implemented genuine Lee-Ready (midpoint-based) separately; fixed the flat-tick fallback to reuse the last known direction; added a tape that actually exercises the tick-test branch and shows EMO/Lee-Ready can disagree. Documented the optimistic zero-queue-ahead bias for orders placed away from the touch (Scenario E, measured) and corrected the "exact single-level replacement" wording to price/size only, not queue position. - Moved tests/ to the repository's top-level tests/ directory (the only blueprint that had its own tests/ subdirectory, which collided with the top-level tests package during unittest's bare discovery and so never ran in CI); made numpy a lazy import so the pure-Python EMO/tick-test classification tests run unconditionally instead of only "skipping cleanly". LOW: - A backtest-only, all-targets Rust build fails on six examples, not four; two (algo, custom_evhandling) are broken independent of the `live` feature or libclang, so installing libclang alone would not get a full `cargo test -p hftbacktest` to pass. Recorded the absence of rust-toolchain.toml/Cargo.lock upstream and this run's resolved versions (cargo tree, generated Cargo.lock hash). - Added Scenario F: NoPartialFillExchange and PartialFillExchange genuinely diverge once order size exceeds the touch's quoted size; the cross-check plan now picks PartialFillExchange deliberately instead of calling the two equivalent. - Relabeled the 69.2ms latency figure as the sensitivity point it is, not a calibration, and planned a sweep instead. - Re-checked the maintenance precondition via the GitHub API: upstream's default branch is unchanged since the already-reviewed 2025-12-23 commit; py-v2.4.4 remains the latest tag. - Reworded wheel provenance (PyPI-served digest match, not independent attestation); fixed the package count (43, not 42) and other nits. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ions Adds Scenario I: PartialFillExchange never depletes self.depth from a fill (three back-to-back 100-share IOCs against an always-100 touch all fill in full, position 300), matching upstream's own doc comment and the reviewer's probe exactly. Corrects the cross-check plan to require matching (or an explicit per-engine difference table for) liquidity/queue settings: reads the actual sim-capacity branch (claude/sim-capacity-20260925, via git show, not touched) confirming liquidity_consumption=True/queue_position=True and the cited 180/900 fills-per-minute budgets, versus sim-paper-compare's liquidity_consumption=False/queue_position=False; picks PartialFillExchange deliberately as the closest-available (not equivalent) configuration. Corrects the NautilusTrader taker-queue claim against the actual rc5 matching_engine/mod.rs source (snapshot_queue_position is only called on the passive/maker branch, never for a taker fill) instead of an unverified paraphrase. Fixes "coincide on A-D and G" to "A-D and G1/G3/G4" (G2, the oversized FOK, genuinely diverges). Corrects the NBER 14158 citation (Asquith, Oman & Safaya 2008, a short-sale misclassification study, not "Diether, Lee & Werner" nor a literature review). Restricts the EMO rule to exact-quote equality (was >=/<=), with new boundary tests confirming a print beyond either quote now falls to the tick test. Actually runs EMO inference on the D1 trade-through tape (with a seed trade so the tick test has history) instead of asserting the result unrun. Nits: FOK/IOC now imported from hftbacktest.order instead of hardcoded; upstream publishes via maturin-action, not twine; the old test directory's CI-invisibility is attributed to missing __init__.py under blueprints/, verified with a minimal reproduction, not a name collision; algo.rs's build failure is a Cargo examples/ packaging issue (a shared helper with no main, also auto-discovered as its own example), not API drift like custom_evhandling's; the "status understates the fill" caveat is scoped to IOC only (FOK expires with exec_qty exactly 0); Scenario E is relabeled hand-verified with its full trace, since ProbQueueModel's early-return path never touches the probability formula here. Re-runs pip-audit with --disable-pip --no-deps, now covering all 43 packages (was 42). Corrects the stale "removal_commands_not_yet_run" cleanup record to reflect the cleanup already executed, and records this round's own rebuild-then-delete of the venv and gives rebuild commands in place of the deleted activate path. Reframes the maintenance check as a dormancy risk (~9 months, 17 open issues) with a concrete 2026-12-23 no-further-commits overturn checkpoint, rather than reading as a pass. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The cross-check plan and receipt had described the sim-capacity engine configuration as of 0e0740e, which fed trade ticks into the engine. From f47160b onward that runner feeds quotes only: in rc5, L1 trade ticks overwrite both book sides, so 41% of IOC fills beat the NBBO when trades were fed. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub. |
… docs - NautilusTrader's liquidity_consumption keeps a per-level consumed-size tally that resets when displayed size changes; it does not decrement the book. - 180-900 is the sim-capacity submission budget per minute, not a fill rate. - The cleanup log records commands and dates only; no exit codes were recorded. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…t; note the quotes-only change Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
deleted the
claude/hftbacktest-crosscheck-20260925
branch
September 25, 2026 06:15
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 25, 2026
…y branch evidence.json was rebuilt from main's copy. This branch's convergence-record entry and its ten sim-capacity file registrations were re-applied. The CI lock path this branch still carried was not re-registered, because #233 renamed it. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 25, 2026
… SIP quotes, native rate limits and full US-equity fees (#235) A sim-lane capacity run, labelled infrastructure evidence and not strategy evidence. It replays real Alpaca SIP data for 8 symbols, 2026-09-24 14:00-14:30Z, through NautilusTrader 2.0.0rc5's simulated exchange. - Setup: a quotes-only engine feed (rc5 L1 trade ticks overwrite both book sides); liquidity consumption; 70 ms latency plus a sweep; SEC, TAF, CAT and Elite commission fees; the native rate limiter at 180/min (paper parity) and 900/min (Elite). - Elite results: 859/873/882 fills/min, with all 30 of 30 minutes at or above 180; 0 fills beating the touch; 0 rejects; the NautilusTrader recount agrees; about $243/min simulated cost. - Paper parity: 171/175/180 fills/min. - Convergence record: NautilusTrader is the selected engine, hftbacktest is the cross-check (#232), and the alternatives are rejected with overturn conditions. Four rounds of Claude review (Codex at its usage limit until 2026-09-30); all 31 mutation probes are caught. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This installs hftbacktest 2.4.4 (Rust 0.9.4, MIT) as the independent cross-check engine for the simulation lane. The choice comes from the 2026-09-25 research convergence. The repo's own record (
manifest-20260922.json) had deferred hftbacktest until intraday execution realism mattered. The 180–900 fills/min sim-capacity lane (blueprints/us-equities/sim-capacity/, in a separate PR) meets that condition. It sits alongside the selected NautilusTrader 2.0.0rc5 engine and does not replace it. Everything is inblueprints/us-equities/sim-crosscheck-hftbacktest/.requirements.lockhash-pinned byuv pip compile --generate-hashes: 43 packages, including hftbacktest 2.4.4 with wheel sha2567df29f3f…, which matches PyPI.uv pip sync --require-hashes.pip-audit --disable-pip --no-depsfound 0 findings on both the PyPI and OSV databases..github/osv-scanner-lockfiles.json.cargo test -p hftbacktest --no-default-features --features backtest --libpasses 22/22. These tests cover the L3 FIFO queue and the depth implementations, not the exchange fill logic.livefeature.NoPartialFillandPartialFillare shown to diverge on oversized orders.Review
Independent Claude review, three rounds. Round 1 found a CI blocker, where a lockfile was not registered with osv-scanner, plus overclaims. Round 2 found wording issues. The final round confirmed:
The Codex CLI is at its usage limit until 2026-09-30, so this PR has no cross-family review.
Checks
tests/test_sim_crosscheck_hftbacktest.pyruns on system python (6 pure-Python tests pass, 17 skip cleanly) and in the venv (23/23 pass).test_osv_lockfile_coveragepasses.🤖 Generated with Claude Code