Skip to content

Memory status shows inconsistent file count due to checkpoint files not being indexed #69759

Description

@xinzf

Bug Description

When running openclaw memory status, the displayed file count shows "Indexed: 3/4 files" even when the indexing is complete. This is because checkpoint session files (created during compaction) are counted in the scan but never actually indexed.

Observed Behavior

Memory Search (coder)
Indexed: 3/4 files · 63 chunks
By source:
 memory · 2/2 files · 2 chunks
 sessions · 1/2 files · 61 chunks  <-- shows 1/2, but checkpoint won't be indexed

The sessions directory contains:

  • 8517049f-75f0-4932-a13d-aa954d33dd89.jsonl (main session file - indexed)
  • 8517049f-75f0-4932-a13d-aa954d33dd89.checkpoint.{UUID}.jsonl (checkpoint - not indexed)

Root Cause Analysis

1. Scan logic counts all .jsonl files

In cli.runtime-D2BW7mgO.js:282 (compiled from src/cli/runtime.ts):

totalFiles: (await fs.readdir(sessionsDir, { withFileTypes: true }))
  .filter((entry) => entry.isFile() && entry.name.endsWith(".jsonl"))
  .length,

This includes checkpoint files (format: {sessionId}.checkpoint.{UUID}.jsonl).

2. Checkpoint files are never indexed

Checkpoint files are created during session compaction (model-context-tokens-z5hvDVkk.js:103):

const snapshotFile = path.join(parsedSessionFile.dir, 
  `${parsedSessionFile.name}.checkpoint.${randomUUID()}${parsedSessionFile.ext || ".jsonl"}`);
fs.copyFileSync(sessionFile, snapshotFile);
// No emitSessionTranscriptUpdate(snapshotFile) is called here

Session files are indexed via event system (manager-cQ8cHF3H.js:1354):

this.sessionUnsubscribe = onSessionTranscriptUpdate((update) => {
  this.sessionsDirtyFiles.add(sessionFile);  // Only files with events are indexed
});

Since checkpoint files don't emit events, they never enter sessionsDirtyFiles and are skipped during sync (manager-cQ8cHF3H.js:1606):

if (!indexAll && !this.sessionsDirtyFiles.has(absPath)) {
  return;  // Skip files not in dirty set
}

3. Design intent

Checkpoint files appear to be temporary snapshots for compaction recovery, not intended for memory indexing. However, the scan logic incorrectly counts them.

Suggested Fix

Modify the scan logic to exclude checkpoint files when calculating totalFiles:

// In src/cli/runtime.ts (scanSessionFiles function)
totalFiles: (await fs.readdir(sessionsDir, { withFileTypes: true }))
  .filter((entry) => 
    entry.isFile() && 
    entry.name.endsWith(".jsonl") && 
    !entry.name.includes(".checkpoint.")  // Exclude checkpoint snapshots
  )
  .length,

This would make the display consistent: "Indexed: 3/3 files" instead of "Indexed: 3/4 files".

Alternative Solutions

  1. Keep current behavior - Accept that checkpoint files are counted but not indexed (current state)
  2. Exclude checkpoint in scan - Simple fix for display consistency (recommended)
  3. Add checkpoint indexing - Emit events for checkpoint creation (larger change, may not be desired design)

Environment

  • OpenClaw version: 2026.4.15 (from openclaw status)
  • Platform: macOS (Darwin 24.6.0 arm64)
  • Node: v24.15.0

Additional Context

This was discovered during investigation of memory indexing behavior. The checkpoint file has valid content (248 messages) and passes isUsageCountedSessionTranscriptFileName() check, but the event-based indexing system intentionally skips it due to lack of session transcript events.

Related code locations:

  • Scan logic: src/cli/runtime.tsdist/cli.runtime-D2BW7mgO.js:282
  • Checkpoint creation: src/gateway/session-compaction-checkpoints.tsdist/model-context-tokens-z5hvDVkk.js:103
  • Session event system: src/plugins/memory-runtime.tsdist/manager-cQ8cHF3H.js:1354
  • Sync filtering: dist/manager-cQ8cHF3H.js:1606

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions