Repository navigation
fix(chroma): keep doc_type out of the semantic search fallback query - #5355
Conversation
[Medium impact] The PR appears safe to merge, with non-blocking extra-query costs remaining. FindingsSummaryAdds a fallback that leaves
Diagram%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Unfiltered over-fetch] --> B{Enough matching rows?}
B -->|Yes| R[Return rows]
B -->|No| C[Try scoped row IDs]
C -->|Answered| R
C -->|Unavailable or insufficient| D{Can remove doc_type safely?}
D -->|No| P[Run full filter]
D -->|Yes| E[Query remaining clauses]
E --> F{Enough rows or short window?}
F -->|Yes| R
F -->|No| P
Reviews (3) · Last reviewed commit: "fix(chroma): keep the doc_type-free fall..." · Reviewed by Greptile |
| const fallback = await this.queryDocTypeFreeFallback( | ||
| runQuery, limit, predicate, stripped.clause, progress | ||
| ); |
There was a problem hiding this comment.
Searches fetch candidates twice
When a doc_type-only filter misses the fast path, this branch repeats the same unfiltered query and applies the same predicate. For limit <= 2000, the window size is identical too. The added test explicitly expects both calls.
Reuse the first window when stripping leaves no clause. A short window already contains every match; a full window with too few matches already requires the plain query. This avoids an extra Chroma call and embedding request.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…d exhaustion (thedotmack#5354) The fallback's null return -- a full stripped window holding too few matches, so only the plain query can prove the top results -- and its progress.exhausted reporting for full and short windows had no test coverage. Logic unchanged.
…udget A search widens its limit in steps, and past the over-fetch cap every step asks the same stripped window. A window that already held too few matches cost another 2000 fragments per step, which pushed the observation search past its 13100 fragment budget (14400 in the budget test). Remember those windows per query, clause and window size and skip straight to the plain filter. Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
Hi @0xABCD01, I pushed two commits to your branch so this can land:
|
| const windowKey = JSON.stringify([query, where, nResults]); | ||
| const knownMatches = this.thinDocTypeFreeWindows.get(windowKey); | ||
| if (knownMatches !== undefined && knownMatches < limit) return null; |
There was a problem hiding this comment.
Search categories share wrong counts
thinDocTypeFreeWindows stores a count after applying the full predicate, but its key includes only the stripped clause. Summary and observation searches with the same query, project and window size therefore share a count despite keeping different documents.
A sparse summary search can make a later observation search skip a window that would satisfy it. Results remain correct, but the search unnecessarily sends the expensive full doc_type filter. Include the original filter in the cache key.
|
Merged, thank you @0xABCD01! When semantic search falls back on a large project, it no longer sends the broad This was checked with mocked Chroma in tests and in CI, not against a store of the size in #5354. It does not change the separate |
Fixes #5354
When the over-fetch leaves fewer than
limitrows and the narrowed row-id fallback cannot run — the scope has more than 5,000 rows, the listing fails, or the caller passes no listing (HybridSearchStrategy) —queryChromasends the caller's fullwhereto Chroma. Every search clause carriesdoc_type, and on a large store that member matches most of the collection (2.85M of 3.09M documents in #5354): chromadb evaluates it in full before intersecting, so one such search blocks chroma-mcp for minutes.The fallback now strips the
doc_typeclause before sending and applies the full clause client-side to the returned metadata — the same split #4687 uses for fragment lookups. The remaining clauses (project scoping, platform) still go to Chroma. The stripped query over-fetches with the fast path's factor and cap (20×, 2,000); a short window proves the match set complete, and a full window with too few survivors still falls through to the plainwhere, so the top-limitguarantee is unchanged. Adoc_typeleaf under an$oris one side of a union and cannot be dropped without changing the match set, so that shape keeps the plain query. The same holds for an$orbranch whose conjuncts are alldoc_type: emptying it would drop that side of the union, so that shape blocks the rewrite and keeps the plain query too.Tests:
tests/services/sync/chroma-query-postfilter.test.tsgains a #5354 block pinning the sent clauses (nodoc_typein any call, residual project/platform clauses preserved, results identical to the plain filter, the$orshapes unchanged), and three #5091 tests that expected the plain fallback now expect the doc_type-free one.bun test tests/services/sync→ 235 pass / 0 fail (230 on main); the search suites (tests/worker/search, search-manager, category arrays, corpus scope, scope row ids) → 197 pass / 0 fail, same as main.bun run typecheck:rootclean.One possible follow-up, not included here: the chunked
sqlite_idlookup past the 5,000-row cap floated in #5354 ("may hold up better"). It changes theChromaFallbackRowIdscontract and multiplies MCP calls per search; this PR is limited to keepingdoc_typeout of the query the fallback sends.The committed bundles are not rebuilt, as in other contributor PRs.