Skip to content

Sim-lane capacity run: ≥180 fills/min on NautilusTrader rc5 with real SIP quotes, native rate limits and full US-equity fees - #235

Merged
seathatflowsinourveins merged 5 commits into
mainfrom
claude/sim-capacity-20260925
Sep 25, 2026
Merged

seathatflowsinourveins merged 5 commits into
mainfrom
claude/sim-capacity-20260925

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator

What

This is a sim-lane capacity run on NautilusTrader 2.0.0rc5's simulated exchange, using real Alpaca SIP data. It shows the simulated execution path can sustain 180 or more fills per simulated minute under realistic execution.

This is infrastructure evidence (sim_capacity_infrastructure), not strategy evidence. It follows NS-06 (docs/harness-rules-convergence-20260922.md), "No strategy manufactures trades" (adaptive-paper/README.md), and mover-v3's "a high-rate capacity replay is an infrastructure test, never strategy evidence". The CapacityExerciser has no signals.

Setup

  • Data: SIP quotes for SPY, QQQ, IWM, AAPL, MSFT, NVDA, AMD and TSLA, 2026-09-24 14:00–14:30Z. They are cached privately, and the receipt records the hashes of the 201 pages this run used. Trades are fetched and catalogued, but the matching engine gets quotes only. In rc5, an L1 trade tick overwrites both book sides, and with trades fed in, 41% of fills beat the NBBO. With quotes only, none of 26,145 fills beat the touch, and a guard asserts this on every run.
  • Venue: L1, liquidity_consumption on, StaticLatencyModel at 70 ms. Latency is not calibrated: the paper broker's observed submit-to-fill was 0.65–1.09 s, so there is a sweep over 0, 70, 250 and 1000 ms.
  • Fees:
    • SEC Section 31 is $20.60 per $1M on sells.
    • FINRA TAF is $0.000195 per share, with a $9.79 cap per execution (FAQ A200.17).
    • The CAT fee is $0.000003 per share on buys and sells.
    • The elite profile adds the Elite all-in commission of $0.0040 per share, cited from Alpaca's fee-schedule PDF.
    • The rounding difference from Alpaca's method of rounding up daily totals is computed and reported.
  • Native rate limiter (RiskEngineConfig.max_order_submit_rate):
    • paper-parity at 180/00:01:00, which is 0.9 × Alpaca's 200 requests per minute, so fills can't exceed 180 per minute;
    • elite-tier at 900/00:01:00, which is 0.9 × 1000. The exerciser attempts about 1,000 per minute, so the limiter binds.
  • Sell sizes are inventory-aware and account for sells already in flight, so there are 0 short-sale rejects.

Results (receipts/20260924-*.json)

Profile Fills per minute (min / median / max) Full minutes ≥180 Cost per minute (all fees) Limiter denials Rejects Fills beating the touch
elite-tier 859 / 873 / 882 30/30 $243.45 2,995 0 0 / 26,145
paper-parity 171 / 175 / 180 1/30 (capped by design) $45.33 3,598 0 0 / 5,255
  • Elite latency sweep: 0 ms gives 889/898.5/900, 250 ms gives 801/829/852, and 1000 ms gives 670/728/757. All 30/30 minutes stay at 180 or above, with 0 violations and 0 rejects at every point.
  • Recount: NautilusTrader's own generate_fills_report() agrees with the exerciser's counters in every run, and cmd_run raises an error on any disagreement.
  • Cost: high trade rates are expensive, as mover-v3 recorded, at roughly $243 per minute at the elite rate.

Research convergence (experiment.json, validated by scripts/validate_convergence.py)

  • Selected: NautilusTrader 2.0.0rc5. It is the same engine as production, and it combines L1/L2/L3 matching, liquidity consumption, latency, halts, native rate limits and custom fee models.
  • Independent cross-check: hftbacktest 2.4.4 (hftbacktest 2.4.4 as the independent sim-lane cross-check engine: pinned install, upstream tests, L1 feasibility #232). The next phase compares the two engines on this data.
  • Rejected, with reasons: Lean, vectorbt/vectorbt.pro, ABIDES, mbt_gym, backtrader, zipline-reloaded, hummingbot, freqtrade and MarS. The candidates came from the user's starred repos, three awesome lists and upstream research on 2026-09-25.
  • Overturn conditions:
    • a 6.5 h session at 1000 fills per minute takes more than 1 h to replay;
    • a v2 position-copy cost regression;
    • divergence from hftbacktest caused by a matching bug;
    • 2.0 final changing queue or latency semantics.
  • Documented rc5 limitation: consumed liquidity at a price level replenishes only when the displayed size changes. This caused 66 of 855 elite IOC expiries at 70 ms, about 0.25% of fills, and biases results pessimistically.

Review

Four rounds of independent Claude review:

  • Reviewers reproduced every headline exactly from --replay.
  • They found the trade-print fills inside the NBBO, stale receipts that had been edited by hand, a limiter that never bound, uncalibrated latency, short-sale rejects on the cash account, and gaps in test protection.
  • All 31 of the reviewer's mutation probes are now caught.

The Codex CLI is at its usage limit until 2026-09-30, so there is no cross-family review.

Checks

  • Tests: tests.test_sim_capacity passes 62/62 on the pinned runtime, and on system python 37 pass with 25 skipping cleanly.
  • Validators: all six pass, and validate_convergence passes.
  • Secret scan: gitleaks history and dir scans are clean.
  • Privacy: no private data, credentials or personal paths are committed.

🤖 Generated with Claude Code

Scout and others added 5 commits September 25, 2026 00:33
…astructure test

Builds blueprints/us-equities/sim-capacity: a real-Alpaca-SIP-data
BacktestEngine capacity exerciser (no signal, no manufactured trades) that
measures fills/simulated-minute under a native RiskEngineConfig rate limiter,
a calibrated StaticLatencyModel, full L1 execution realism (trade_execution,
liquidity_consumption, queue_position) and a cited SEC/FINRA fee model.
paper-parity (180/min budget) stays capped as designed; elite-tier (900/min)
sustains >=180 fills/min in every full minute across a 0/70/250/1000ms
latency sweep, independently recounted from NautilusTrader's own fills
report. Evidence is tagged sim_capacity_infrastructure throughout: this is
capacity evidence, never strategy or broker-throughput evidence.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
… sides, full fee stack

Independent review found the pinned rc5 engine's L1 book gets locked to a
trade print's price with nothing restoring the prior quote for a
NO_AGGRESSOR trade, letting 41.2% of elite-tier IOC fills beat the NBBO
touch. Fixed by feeding QuoteTicks only into the BacktestEngine (trade ticks
stay in the private ParquetDataCatalog for future passive-order work but are
no longer replayed here); measured 0 touch violations after the fix, now
asserted by cmd_run and covered by a dedicated test
(check_no_fill_beats_nbbo_touch). queue_position/trade_execution are
documented as having no measured effect in this quotes-only, IOC-only setup;
only liquidity_consumption does.

Side selection is now inventory-aware (schedule.RoundRobin never proposes a
sell from a flat/short position; sell qty is clamped to the current long),
eliminating the CASH-account short-sale rejects that previously consumed
budget and drove position-cap skips (162/163 and 112/113 rejects in one run).
Elite-tier cadence now exceeds its budget (~1000/min vs 900/min, not exactly
900/min) so its rate limiter actually binds and denials are counted.
PRIMARY_LATENCY_MS is documented as a primary reference point, not a
calibration, with the measured submit-to-fill distribution (which mixes
configured latency with the exerciser's own timer cadence) reported per run.

Fee model adds FINRA CAT (buys and sells) and an optional Alpaca Elite Smart
Router commission (elite-tier default: $0.0040/share all-in), both cited
from Alpaca's dated Brokerage Fee Schedule PDF. Regenerated both receipts
with the committed runner (runner_sha256 now matches); fixed the receipt's
page-hash accounting to record exactly this run's served pages, separately
from the cumulative private-cache ledger; excluded the flatten pad from
reported quote/trade counts; and fixed a real unit-forcing gap in
dataframe_to_rows (a datetime64[us] column would have silently produced
microseconds, not nanoseconds).

Added tests calling runner.run_one directly (the real production venue/risk/
fee config, not a hand-built engine) plus targeted unit tests, closing every
mutation gap an independent mutation-probe harness found in the prior round.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ervation, TAF settled

TAF cap scope is settled (FINRA TAF FAQ A200.17: per execution, not per
order); removed the UNVERIFIED flag from fee_model.py, README.md, runner.py
and experiment.json, citing that source.

MEDIUM fixes:
- The NBBO-touch guard compared each fill against only the last quote at its
  timestamp, producing a false positive when two quotes share a nanosecond
  (1,342 such pairs in the real data). It now checks every quote sharing the
  in-force timestamp and only flags a fill that beats all of them. The guard
  is now asserted on every run, including every latency-sweep point (it
  previously covered only the primary runs).
- Added tests for the H1 fix and its guard: a run_one test with trades
  present (including a sub-penny print inside the NBBO, injected mid-gap in
  sparse quotes so the rc5 book-overwrite artifact actually persists to a
  matching instant) asserts the engine feed excludes them; a separate test
  injects a synthetic violation and asserts the guard detects it.
- Concurrent in-flight sells for the same symbol were each clamped against
  the same stale, not-yet-reduced position, letting them collectively
  request more than was held (58 unreported short-sale rejects in the
  1000ms-latency sweep). Sells now reserve their quantity per symbol at
  submit time, released on any terminal outcome; rejects are reported by
  reason in every run, including the sweep (now genuinely zero throughout).
- Rewrote the inventory-rule test that previously proved nothing (it forced
  _last_side="SELL", so plain alternation already returned BUY) to force
  _last_side="BUY" instead, the actually discriminating case.

Added tests closing every new-mutation gap the reviewer's delta_mutate.py
found (trade ticks re-added, guard disabled, cmd_run at 0ms latency, run_one
ignoring the profile budget, a sell from flat, the sell clamp removed, the
elite commission dropped, a 10s recount-start shift), and replaced two
literal-pin checks with behavioral ones where feasible (a real run's
denial count against the documented budget; the real StaticLatencyModel
wiring), keeping only the source-level check for realism flags with no
independent behavioral signal (queue_position/trade_execution, documented as
no-op in this quotes-only, IOC-only setup).

LOW fixes: corrected the latency framing (70ms is far BELOW the observed
0.65-1.09s paper range, not inside it); the fee-rounding difference vs
Alpaca's aggregate-then-round-up method is now computed per run from the
actual fills (runner.alpaca_rounding_delta) and reported in the receipt,
not asserted as a fixed note; the Elite all-in rate is documented as flat
across every volume tier, not "<=200,000"; reworded the "independently
re-verified" claim (the builder ran the reviewer's harness as a cross-check;
the reviewer verifies independently); fixed experiment.json's scope strings
to say quotes-only; cmd_run now raises explicit errors instead of asserts
(which -O would silently disable); reworded the Elite request-rate mentions
to be explicit that 1000 calls/min is a request budget, not a throughput
figure.

Re-ran both profiles and the full latency sweep on real Alpaca SIP data:
headlines reproduce exactly (elite 859/873/882, paper 171/175/180), zero
NBBO-touch violations and zero rejects throughout, recount agrees on every
run. Regenerated both receipts with the committed runner (runner_sha256
verified to match) and updated experiment.json/evidence.json hashes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…, X6 clamp test

- Fixed a stale README sweep row (1000ms: 655/724.5/758) that no longer
  matched the committed receipt (670/728.0/757). Added
  test_readme_latency_sweep_table_matches_receipt (parses both and asserts
  every row agrees) so this cannot silently drift again.
- Documented an undocumented rc5 behavior: a book level's consumed-quantity
  tally under liquidity_consumption=True resets only when that price's
  displayed size actually changes; an identical repeated (price, size) quote
  never replenishes. Measured on the committed elite-tier@70ms receipt: 66 of
  855 IOC expiries (~0.25% of the 26,155 total fills) are this artifact,
  making the lane's numbers pessimistic, not inflated. Added a README
  Limitations section and a matching experiment.json limitation entry. Fixed
  the shared RealVenueRunOneTests fixture (previously a constant, unvarying
  quote that measurably locked up after 4 fills, leaving every behavioral
  assertion in that class resting on those same 4 fills) to cycle displayed
  size 3..7 tick-to-tick instead.
- Added an engine-level test for the qty clamp itself (not just the sell
  reservation, which already had engine coverage): an asymmetric ask=2..6/
  bid=5..9 fixture where a SELL's displayed-size-derived desired quantity
  habitually exceeds a thin position. Verified directly: disabling the clamp
  on this fixture produces 44 short-sale rejects; the real clamp produces
  zero. Fixing the shared fixture's liquidity had silently broken the
  existing reservation-race test's own coverage (it now relies on its own
  dedicated, deliberately liquidity-locked fixture instead, restoring that
  coverage) -- caught and fixed by re-running the reviewer's mutation probes
  before finishing, not just once at the start.
- Fixed the remaining "quotes/trades" scope strings in experiment.json (the
  engine is fed quotes only).
- Replaced the paraphrased, non-verbatim FINRA TAF FAQ A200.17 quote with the
  actual text (a member "may choose to calculate the Trading Activity Fee on
  either the individual street side executions or on the account level
  average price confirmation"), correctly scoped to average-price-allocated
  orders, with the per-execution conclusion kept for this exerciser's plain
  per-fill activity (which does no such allocation).
- Fixed a temp-directory leak in the cmd_run integration test (now uses
  TemporaryDirectory with addClassCleanup instead of a bare mkdtemp).

Re-ran both profiles and the full latency sweep on real Alpaca SIP data:
headlines unchanged (elite 859/873/882, paper 171/175/180), zero touch
violations and zero rejects throughout. Regenerated both receipts (runner.py
text changed) and verified runner_sha256 matches exactly. Re-ran the
reviewer's final_mutate.py: all 31 probes (14 original + 17 new-code) now
CAUGHT, zero not-caught, zero skipped.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…y branch

evidence.json was rebuilt from main's copy. This branch's convergence-record
entry and its ten sim-capacity file registrations were re-applied. The CI lock
path this branch still carried was not re-registered, because #233 renamed it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@seathatflowsinourveins
seathatflowsinourveins merged commit 1e11a89 into main Sep 25, 2026
24 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/sim-capacity-20260925 branch September 25, 2026 07:25
seathatflowsinourveins added a commit that referenced this pull request Sep 25, 2026
…run (#235) (#238)

The simulation lane now reads 859 to 882 fills/min under the elite budget, with
all 30 of 30 minutes at or above 180, and 171 to 180 at paper parity. The engine
is fed quotes only, and the run is infrastructure evidence. Its evidence_ref
points to the elite-tier receipt.

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant