Skip to content

fix(sqlite): skip empty-title observations at capture - #3176

Merged
thedotmack merged 1 commit into
thedotmack:mainfrom
winklemad:fix/skip-empty-title-observations
Sep 11, 2026
Merged

thedotmack merged 1 commit into
thedotmack:mainfrom
winklemad:fix/skip-empty-title-observations

Conversation

@winklemad

@winklemad winklemad commented Jul 8, 2026 •

Copy link
Copy Markdown
Contributor

What & why

content_hash dedup only catches byte-identical observations, so it can't stop malformed rows from piling up. #3163 measured 212 empty-title observations in a real ~45k-row DB. They're pure noise that still take slots in the recency-ordered recall window injected at session start.

This is the smallest, no-ambiguity slice of #3163: stop empty-title observations at capture.

Change

  • SessionStore.storeObservations (the live capture path the summarizer uses) skips any observation whose title is null, empty, or whitespace-only, so the row is never written.
  • The single-observation storeObservation wrapper requires a non-empty title (throws rather than returning an undefined id), keeping its { id: number } contract.
  • Its one caller, POST /api/memory/save (MemoryRoutes), now falls back to the text-derived title for empty and whitespace-only titles. Previously title || fallback let a whitespace-only title through, which would have hit the guard; text is validated non-empty, so the fallback is always valid.
  • importObservation is untouched, so restoring an existing DB preserves historical rows.

Testing

  • tests/sqlite/session-store-empty-title.test.ts: batch skips empty/whitespace/null titles; the single-observation wrapper throws on an empty title and stores a titled one.
  • tests/worker/http/routes/memory-routes.test.ts: a whitespace-only title falls back to the text-derived title, and a provided title is used (trimmed).
  • SQLite suite: 72 pass. Route suite: 9 pass. tsc --noEmit: clean.

Follow-ups

The bigger part of #3163 (coalescing repeated (project, title) boilerplate onto a counter) has real design choices: exact vs fuzzy title match, which types to coalesce, and the coalesce semantics. I left it out to keep this bounded. Happy to do it next if you let me know how you'd want the dedup to behave.

Refs #3163

@greptile-apps

greptile-apps Bot commented Jul 8, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR stops empty-title observations from being stored during live capture. The main changes are:

  • Batch observation storage skips null, empty, and whitespace-only titles before inserting rows.
  • Single-observation storage now throws when called with an empty title instead of returning an undefined id.
  • Manual memory saves trim provided titles and fall back to a text-derived title for whitespace-only input.
  • Tests cover the SQLite storage behavior and the HTTP route title fallback.

Confidence Score: 5/5

Safe to merge with minimal risk.

The change is narrowly scoped to empty-title observation handling. The previous whitespace-title route issue is addressed by trimming before fallback. Tests cover the relevant storage and HTTP paths.

No files require special attention.

T-Rex T-Rex Logs

What T-Rex did

  • Validated the SQLite observation of empty-title behavior in the session store using the empty-title behavior log.
  • Validated how the memory save route handles whitespace in titles by reviewing the whitespace-title fallback log.
  • Verified the root TypeScript typecheck results using the typecheck-root log.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
src/services/sqlite/SessionStore.ts Adds empty-title validation for single observation storage and skips null/blank-titled rows in batch capture before insert/dedup.
src/services/worker/http/routes/MemoryRoutes.ts Trims provided manual-memory titles and falls back to text-derived titles when omitted or whitespace-only.
tests/sqlite/session-store-empty-title.test.ts Covers batch skipping of empty/null/whitespace titles and single-observation behavior for rejected and accepted titles.
tests/worker/http/routes/memory-routes.test.ts Adds route coverage for whitespace-only title fallback and trimming of provided manual-memory titles.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Client
participant MemoryRoutes
participant SessionStore
participant SQLite

Client->>MemoryRoutes: POST /api/memory/save(text, title?)
MemoryRoutes->>MemoryRoutes: trim title or derive title from text
MemoryRoutes->>SessionStore: storeObservation(observation)
SessionStore->>SessionStore: reject empty/blank title
SessionStore->>SessionStore: storeObservations([observation])
SessionStore->>SQLite: insert titled observation

Note over SessionStore,SQLite: Batch capture skips null/empty/whitespace titles before INSERT
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Client
participant MemoryRoutes
participant SessionStore
participant SQLite

Client->>MemoryRoutes: POST /api/memory/save(text, title?)
MemoryRoutes->>MemoryRoutes: trim title or derive title from text
MemoryRoutes->>SessionStore: storeObservation(observation)
SessionStore->>SessionStore: reject empty/blank title
SessionStore->>SessionStore: storeObservations([observation])
SessionStore->>SQLite: insert titled observation

Note over SessionStore,SQLite: Batch capture skips null/empty/whitespace titles before INSERT
Loading

Reviews (3): Last reviewed commit: "fix(sqlite): skip empty-title observatio..." | Re-trigger Greptile

@winklemad
winklemad force-pushed the fix/skip-empty-title-observations branch from c77e08f to fdf0098 Compare July 8, 2026 17:24
Comment thread src/services/sqlite/SessionStore.ts Resolved
Empty-title observations are malformed, low-signal rows that fill the recency-based recall window without adding any facts. Skip them in storeObservations so they stop accumulating in the observations table.

The single-observation storeObservation wrapper now requires a non-empty title (throws instead of returning an undefined id), and its /api/memory/save caller falls back to the text-derived title for whitespace-only titles so a blank title never reaches the store.

Refs thedotmack#3163
@thedotmack

Copy link
Copy Markdown
Owner

Routing note from the PR consolidation pass. Every open PR was read in full and clustered by root cause; this one is tracked under #3606 ([plan-18] Observer Response Pipeline) as standalone / merge-now: it is small and self-contained enough to land on its own ahead of the plan-18 bundle once CI is green.

step 5 (missing title rejected at write time)

The master's roster comment lists the sibling PRs targeting the same root cause and the fix-sequence order — please follow that issue for progress. Thank you for the work here.

@winklemad

Copy link
Copy Markdown
Contributor Author

Thanks @thedotmack — CI is green here (the only check is the skipped Mintlify docs deploy), so this is ready to land whenever it fits the plan-18 sequence. Appreciate the consolidation write-up.

@thedotmack

Copy link
Copy Markdown
Owner

NIGHT SHIP after Batch H — issue-patch-ship: Gate FAIL.

MERGEABLE / CLEAN, but CLEAN is only the skipped Mintlify check — not product CI. Skipping empty-title rows at capture is an opinionated filter (drop data from the recency window), not a crash/corruption RCA. The July hunks also sit on a SessionStore that has moved a long way since. Leaving open; not rebasing in this batch.

@thedotmack
thedotmack merged commit 0bbef55 into thedotmack:main Sep 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants