Repository navigation
fix(sqlite): skip empty-title observations at capture - #3176
Conversation
c77e08f to
fdf0098
Compare
Empty-title observations are malformed, low-signal rows that fill the recency-based recall window without adding any facts. Skip them in storeObservations so they stop accumulating in the observations table. The single-observation storeObservation wrapper now requires a non-empty title (throws instead of returning an undefined id), and its /api/memory/save caller falls back to the text-derived title for whitespace-only titles so a blank title never reaches the store. Refs thedotmack#3163
fdf0098 to
68fb476
Compare
|
Routing note from the PR consolidation pass. Every open PR was read in full and clustered by root cause; this one is tracked under #3606 ([plan-18] Observer Response Pipeline) as standalone / merge-now: it is small and self-contained enough to land on its own ahead of the plan-18 bundle once CI is green.
The master's roster comment lists the sibling PRs targeting the same root cause and the fix-sequence order — please follow that issue for progress. Thank you for the work here. |
|
Thanks @thedotmack — CI is green here (the only check is the skipped Mintlify docs deploy), so this is ready to land whenever it fits the plan-18 sequence. Appreciate the consolidation write-up. |
|
NIGHT SHIP after Batch H — issue-patch-ship: Gate FAIL.
|
What & why
content_hashdedup only catches byte-identical observations, so it can't stop malformed rows from piling up. #3163 measured 212 empty-title observations in a real ~45k-row DB. They're pure noise that still take slots in the recency-ordered recall window injected at session start.This is the smallest, no-ambiguity slice of #3163: stop empty-title observations at capture.
Change
SessionStore.storeObservations(the live capture path the summarizer uses) skips any observation whose title is null, empty, or whitespace-only, so the row is never written.storeObservationwrapper requires a non-empty title (throws rather than returning an undefined id), keeping its{ id: number }contract.POST /api/memory/save(MemoryRoutes), now falls back to the text-derived title for empty and whitespace-only titles. Previouslytitle || fallbacklet a whitespace-only title through, which would have hit the guard;textis validated non-empty, so the fallback is always valid.importObservationis untouched, so restoring an existing DB preserves historical rows.Testing
tests/sqlite/session-store-empty-title.test.ts: batch skips empty/whitespace/null titles; the single-observation wrapper throws on an empty title and stores a titled one.tests/worker/http/routes/memory-routes.test.ts: a whitespace-only title falls back to the text-derived title, and a provided title is used (trimmed).tsc --noEmit: clean.Follow-ups
The bigger part of #3163 (coalescing repeated
(project, title)boilerplate onto a counter) has real design choices: exact vs fuzzy title match, which types to coalesce, and the coalesce semantics. I left it out to keep this bounded. Happy to do it next if you let me know how you'd want the dedup to behave.Refs #3163