Skip to content

[Bug]: Memory-core session indexer skips .jsonl.reset and .jsonl.deleted files — most past sessions invisible to search #57334

Description

@christian-lallo

Bug type

Regression (worked before, now fails)

Beta release blocker

No

Summary

The listSessionFilesForAgent function in query-expansion-CeNhqo71.js filters session files with .filter((name) => name.endsWith(".jsonl")), which excludes .jsonl.reset.* and .jsonl.deleted.* files. After the daily 4 AM session reset, main session transcripts are renamed from <id>.jsonl to <id>.jsonl.reset.<timestamp> — making them invisible to the session indexer. Only small cron/subagent sessions (which aren't reset) and the current live session remain as .jsonl. Result: 211 .jsonl files indexed out of 485 total, with all large main conversation sessions excluded.

Steps to reproduce

  1. Configure an agent with sources: ["memory", "sessions"] and experimental.sessionMemory: true.
  2. Use the agent actively for several days with the daily 4 AM session reset enabled.
  3. Run ls ~/.openclaw/agents/main/sessions/ | grep -c '\.jsonl$' (active files) vs ls | wc -l (total files).
  4. Run openclaw memory index --force --verbose and note the session chunk count.
  5. Run sqlite3 ~/.openclaw/memory/main.sqlite "SELECT path, COUNT(*) FROM chunks WHERE source='sessions' GROUP BY path ORDER BY COUNT(*) DESC LIMIT 10" — only small cron sessions appear.

Source code reference: query-expansion-CeNhqo71.js, line ~413:

.filter((name) => name.endsWith(".jsonl"))

Expected behavior

All session transcript files — including .jsonl.reset.* and .jsonl.deleted.* — should be included in session indexing. These are completed session transcripts that represent the bulk of conversation history. The file filter should match .jsonl anywhere in the filename, not just at the end. e.g. .filter((name) => name.includes(".jsonl"))

Actual behavior

Only 211 out of 485 session files are indexed. The 274 excluded files include all main conversation sessions (the largest and most valuable transcripts). A 13.7MB main session with 416 user/assistant messages (~214K chars of conversation text, estimated ~133 chunks) produces 0 indexed chunks because it was renamed to .jsonl.reset.* after the 4 AM session reset.

The indexed sessions are almost entirely small cron/subagent sessions (1-10 chunks each), giving the false impression that the chunker is "sparse" when in reality the large sessions simply aren't being read.

# Our deployment:
Total files in sessions dir: 485
Files ending in .jsonl: 211 (small cron/subagent sessions)
Files ending in .jsonl.reset.*: 57 (main conversation sessions — EXCLUDED)
Files ending in .jsonl.deleted.*: 217 (archived sessions — EXCLUDED)

# Database shows:
Current live session: 173 chunks (correctly indexed)
Second largest: 10 chunks (small cron session)
All main past sessions: 0 chunks (invisible)

OpenClaw version

2026.3.24 (cff6dc9) — also confirmed in 2026.3.28 source

Operating system

macOS 25.3.0 (Mac mini M4, arm64)

Install method

npm global

Model

N/A — indexing bug, not model-specific

Provider / routing chain

N/A — indexing bug

Additional provider/model setup details

No response

Logs, screenshots, and evidence

Impact and severity

Affected: Any user with daily session resets enabled (the default behavior). All main conversation sessions become invisible to memory-core search after the reset renames them.
Severity: High — effectively disables cross-session memory recall via the native session indexer. Users believe session indexing is working (status shows sources: ["memory", "sessions"]) but past conversations are silently excluded.
Frequency: 100% — every session reset renames the file, every renamed file is excluded.
Consequence: memory_search cannot find content from any past main session. Only cron/subagent sessions and the current live session are searchable. This defeats the purpose of sources: ["sessions"].

Additional information

Related to #44028 (shouldSyncSessions bug) — we initially attributed the sparse session chunk count (380 chunks from 204 files) to aggressive chunking. The real cause is that the 204 .jsonl files are mostly tiny cron sessions. The large main sessions were never in the candidate pool because they'd been renamed to .reset or .deleted.

Our workaround: a custom transcript-to-md.py script that converts ALL session files (including .reset and .deleted) to markdown, which memory-core then indexes as source="memory" files with proper chunking (~24 chunks per file). This works but should not be necessary — the native session indexer should handle renamed files.

Suggested fix: Change the file filter from name.endsWith(".jsonl") to name.includes(".jsonl") (or a regex like /\.jsonl(\.|$)/), so .jsonl.reset.* and .jsonl.deleted.* files are included in session indexing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingregressionBehavior that previously worked and now fails

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions