Repository navigation
perf(embeddings): skip unused BERT suffixes with exact token guards - #496
Conversation
…en-embedding-prefix-audit
…en-embedding-prefix-audit
…en-embedding-prefix-audit
…en-reuse-write-embeddings
cdeust
changed the title
Brings in #503 (floor check non-blocking + --no-regression gate) and #505 (mcp-host-config timeout sized to its measured tail), so this PR is verified against the CI its merge will actually land on. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1
Brings in #506 (docker-smoke cache export skipped on pull_request), the fix for the Docker Smoke timeout that was the only red check on this branch's last run. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1
cdeust
left a comment
cdeust
left a comment
There was a problem hiding this comment.
Completed source review of this implemented scope and its parent integration. No unresolved correctness, security or lifecycle finding. The description records limitations rather than claiming unmet optimization targets. Merge remains gated on current required CI and applicable baseline quality evidence. Review includes independent read-only review of the per-PR diff and combined locked local tests.
cdeust
left a comment
cdeust
left a comment
There was a problem hiding this comment.
Reviewed the CI-trigger commit: git diff from da4a998 is empty and Git tree 8865c2f4c854d13fa56b5536df9922b6433f207e is unchanged. Previous source review and test evidence apply to this identical tree. Required main-targeted checks are now running; no protection bypass is used.
Skip unused BERT suffixes only when the validated tokenizer configuration, whole-word boundary and saturated token budget prove the same model input.
Sixteen measured model inputs and vector byte sequences match exactly. Original text, cache key and batch path remain intact; timings are fixture-specific.
Validation and review (2026-09-08):
2462ca1e4d49f70abd2553194b672e344359a1bf.ca32ba50is running on separate disposable databases with identical public datasets. It uses the repository's existing 0.005 tolerance; no acceptance result is claimed before completion.Historical measurements apply to their recorded revisions and scopes. The current PR is authorized for merge after applicable checks; remaining optimization targets above are not presented as completed benefits.
The latest commit only triggers main-targeted security checks. Its tree
8865c2f4c854d13fa56b5536df9922b6433f207eis identical toda4a99823395476f7eb915446ef228166c973869. The composed benchmark uses frozen commit5000f75e5d1ee14c5dd62db709a99f86792dab2d, whose tree matches final PR #502.