Skip to content

Bug: orphan tool_use persisted to session .jsonl after rate limit retry exhaustion #26936

Description

@HanenVit

Description

When the Anthropic API returns 429 (rate limit) and all retries are exhausted, the Gateway saves the assistant message containing tool_use blocks to the session .jsonl file without a corresponding tool_result block. This creates an invalid message history that permanently blocks the session — every subsequent API request fails with:

messages.N: `tool_use` ids were found without `tool_result` blocks immediately after: execXXX

The session remains dead until the Gateway is restarted (creating a new session).

Steps to Reproduce

  1. Agent (Claude Opus) calls a tool (e.g., exec)
  2. API rate limit (429) is hit during the request
  3. Gateway retries multiple times (observed 20 retries in ~3 minutes)
  4. All retries fail → Gateway saves the assistant message with tool_use to .jsonl
  5. tool_result is never written (the tool was never executed)
  6. Next heartbeat/request → Anthropic API rejects with 400 tool_use without tool_result
  7. Session is permanently broken

Observed Behavior (Feb 25, 2026)

22:11:35-22:14:05 — 20x rate limit retries (same runId)
22:26:33 — First 400 error (orphan tool_use: exec17720470437771)
22:26-00:49 — 7 consecutive heartbeats, all failed with same error
01:05 — Gateway restart → new session (workaround, not fix)

Additional context: Gateway was in a crash-loop (634 restarts) due to an unrecognized config key (cron.fleet-monitor), which likely contributed to the rate limit being hit.

Expected Behavior

If all rate limit retries are exhausted:

  • The orphan tool_use assistant message should not be persisted to the session .jsonl
  • OR a synthetic tool_result with an error message should be appended (e.g., {"type": "tool_result", "content": "Rate limit exceeded, tool was not executed"})

Workaround

We built a cron-based session-healer plugin that:

  1. Detects the error pattern in Gateway logs
  2. Finds the orphan tool_use line in .jsonl by ID
  3. Surgically removes it + injects a recovery message
  4. Escalates (retry → remove compaction summary → rotate session)

Environment

  • OpenClaw: 2026.2.23 (b817600)
  • Model: anthropic/claude-opus-4-6
  • OS: Ubuntu Linux 6.8.0
  • Node: v22.22.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    staleMarked as stale due to inactivity

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions