Skip to content

Overflow recovery duplicates role=user messages in session JSONL, amplifying transcript growth #66443

Description

@holisticseo

Summary

When OpenClaw enters context overflow recovery (for example, logs like [context-overflow-recovery] Attempting tool result truncation), the agent session JSONL can end up with duplicated role=user entries (same content hash and length), even when tool-result bloat is not present in the session file. This amplifies transcript growth and can cascade into additional overflows.

Evidence (one capture)

  • sessionKey: agent:main:main:heartbeat
  • session JSONL contained 3 copies of the same user message (1 unique message):
    • sha256(content) prefix: b266b2bfea51
    • len: 442
    • occurrences: 3
    • createdAt timestamps:
      • 2026-04-14T08:36:16.018Z
      • 2026-04-14T08:40:28.997Z
      • 2026-04-14T08:46:41.910Z
    • content preview: Read HEARTBEAT.md if it exists (workspace context)...

Gateway log correlation (same machine):

  • 2026-04-14T01:40:25.557-07:00 [context-overflow-diag] ... diagId=ovf-mnydfsgk-5y5CJA
  • 2026-04-14T01:40:25.559-07:00 [context-overflow-recovery] Attempting tool result truncation ...
  • 2026-04-14T01:45:06.714-07:00 [context-overflow-diag] ... diagId=ovf-mnydltei-OtPZfw compactionAttempts=1

Expected

Overflow recovery should not duplicate existing user messages in the persisted session store.

Observed

After recovery, the persisted session contains duplicate copies of a prior user message (byte-identical).

Possible repro (hypothesis)

  1. Start a session with a single short user prompt.
  2. Force a context overflow (large tool result or repeated messages) until overflow recovery triggers.
  3. Let recovery complete (tool result truncation / compaction).
  4. Inspect the session JSONL for duplicated role=user entries (dedupe by content hash and len, or by message id if present).

Why it matters

This creates artificial transcript growth that can worsen or re-trigger overflows, especially in long-running sessions.

Suspected root area

Overflow recovery / truncation code path may be re-appending the last user message (or replaying a block) without idempotency or deduping in the session writer.

Ask

  • Confirm and patch overflow recovery/session persistence so prior user messages are not duplicated.
  • Add guardrails (for example, dedupe by stable message id, or a last-N hash window) when writing recovered transcripts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.staleMarked as stale due to inactivity

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions