Repository navigation
fix(search): widen filtered semantic candidates before pagination - #4960
Conversation
Signed-off-by: hwan <[email protected]>
…ager Signed-off-by: hwan <[email protected]>
[Medium impact] The PR appears safe to merge; no blocking issues remain. SummaryRoutes semantic searches through the shared Chroma strategy and widens candidates before pagination.
The earlier finding about full categories hiding sparse categories is fixed: each requested category must now have enough rows before widening stops. No new actionable issues were found. Diagram%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Semantic search] --> B[Fetch candidate window]
B --> C[Apply date and row filters]
C --> D{Every requested category full?}
D -->|Yes| G[Apply final pagination]
D -->|No| E{More candidates within budget?}
E -->|Yes| B
E -->|Work cap reached| F[Keep semantic rows and add keyword matches]
E -->|Documents exhausted| H[Keep existing empty-category fallback]
F --> G
H --> G
Reviews (3) · Last reviewed commit: "test(search): keep work-cap fixtures ins..." · Reviewed by Greptile |
Signed-off-by: hwan <[email protected]>
…candidates before pagination Squashed from thedotmack/claude-mem pull/4960 (source only, CHANGELOG excluded). Claude-Session: https://claude.ai/code/session_01J69ZRK5a82yX2rX1vVhjeo
Seed 2048 excluded rows instead of 8192: just past the 1600-candidate semantic work cap, which is all the scenario needs. Give the two multi-fixture work-cap tests an explicit 20s timeout for slower runners. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01QVujqk6gvQYaoymRd7Aynn
|
Merged, thank you @junnhwan! Filtered semantic searches (date range, type, concepts, files, or the default recency window) no longer lose in-scope matches that ranked past Chroma's first 100 candidates: both search paths now share What we changed:
|
Closes #4959
Filtered Smart searches can return a single surviving row and lose a recent supporting memory at semantic rank 148, even though that row is fully indexed and retrievable by ID. Both unified implementations stopped at the first 100 candidates before date and SQLite row filters.
Route SearchManager's semantic selection through the existing Chroma strategy and apply #4510's bounded widening mechanism there. For interactive budgets up to 1600, widen through 100/200/400/800/1600 only while the filtered candidate page is insufficient; stop on raw-document exhaustion, cache accepted and rejected SQLite lookups by category, and hydrate in batches of 500. At the work cap, retain semantic matches and fill remaining slots from filtered keyword results without duplicate IDs. Partial-page keyword additions honor the active date window; the existing empty-category fallback policy is preserved.
Keep final pagination, relevance/date ordering, project/source scopes, and larger explicitly requested corpus budgets. No schema, setting, backend or distributed bundle changes. Source-only, following the maintainer release-bundle guidance.
Validation:
The finite work budget does not guarantee semantic matches beyond its prefix or bound backend wall-clock latency; #4938's timeout/timeline issue is separate. The real-backend fixture is synthetic, not a production recall or task-quality benchmark. PostgreSQL and installed user caches were not modified.
AI assistance: investigated, implemented and verified with Codex; commits include DCO sign-off.