Skip to content

fix(memory): eliminate 255 redundant BPE loads in proptest#2327

Merged
bug-ops merged 1 commit intomainfrom
fix/slow-proptest-token-counter
Mar 28, 2026
Merged

fix(memory): eliminate 255 redundant BPE loads in proptest#2327
bug-ops merged 1 commit intomainfrom
fix/slow-proptest-token-counter

Conversation

@bug-ops
Copy link
Copy Markdown
Owner

@bug-ops bug-ops commented Mar 28, 2026

Problem

count_tokens_never_panics called TokenCounter::new() inside the proptest body. Each call to TokenCounter::new() parses the tiktoken cl100k_base BPE vocabulary (~1 MB, ~100k HashMap entries). With 256 proptest cases, this resulted in 256 full BPE HashMap constructions per test run instead of one.

The cl100k_base singleton exists in production code exactly to avoid this, but the test bypassed it.

Fix

Use a LazyLock static so the TokenCounter is initialized once per test binary execution — matching how the production agent uses it (one counter per session).

// before
fn count_tokens_never_panics(s in ".*") {
    let counter = crate::token_counter::TokenCounter::new(); // × 256
    let _ = counter.count_tokens(&s);
}

// after
static COUNTER: LazyLock<TokenCounter> = LazyLock::new(TokenCounter::new); // × 1
fn count_tokens_never_panics(s in ".*") {
    let _ = COUNTER.count_tokens(&s);
}

Result

Test runtime: ~several seconds → 0.204s (measured locally).

Bonus

Add slow-timeout = { period = "30s" } warning override to the ci-partition nextest profile as a safety net — nextest prints a warning if any test exceeds 30s without failing the run, surfacing future regressions early.

count_tokens_never_panics called TokenCounter::new() inside the proptest
body, constructing and parsing the tiktoken cl100k_base BPE vocabulary
(~1 MB, ~100k HashMap entries) on every one of the 256 proptest cases.

Replace with a LazyLock static so the TokenCounter is initialized once
per test binary execution, matching how the production agent uses it.

Also add slow-timeout = 30s warning to the ci-partition nextest profile
as a safety net to surface future slow tests in CI output.
@github-actions github-actions bot added memory zeph-memory crate (SQLite) rust Rust code changes ci CI/CD configuration bug Something isn't working size/XS Extra small PR (1-10 lines) labels Mar 28, 2026
@bug-ops bug-ops enabled auto-merge (squash) March 28, 2026 08:42
@bug-ops bug-ops merged commit ab22e8b into main Mar 28, 2026
25 checks passed
@bug-ops bug-ops deleted the fix/slow-proptest-token-counter branch March 28, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci CI/CD configuration memory zeph-memory crate (SQLite) rust Rust code changes size/XS Extra small PR (1-10 lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant