Skip to content

OpenAI-compatible streaming with llama.cpp saves zero usage (stream closed before final usage chunk) #79897

Description

@alexanderatkins

Summary

When using OpenClaw with a llama.cpp backend (openai-completions API, SSE streaming), token usage is always saved as 0/0/0, breaking /status context display and compaction behavior.

Root Cause (diagnosed)

llama.cpp emits a final usage-only SSE chunk after the finish_reason: stop chunk. OpenClaw appears to stop consuming the stream once it sees finish_reason: stop, so the trailing usage chunk is never processed.

Evidence from llama.cpp server log:

new prompt, n_ctx_slot = 120064, task.n_tokens = 17332
stop processing: n_tokens = 17553, truncated = 0

This confirms the model had a valid ~17.3k token prompt. But OpenClaw saves:

"usage": {"input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0, "totalTokens": 0}

The llama.cpp log also shows:

reasoning-budget: activated, budget=2147483647 tokens
srv stop: cancel task, id_task = 112

The cancel task line suggests the client (OpenClaw) closes the connection around finalization, dropping the final usage chunk.

A direct curl of the same endpoint confirms the usage chunk is sent by llama.cpp — it's just not making it into OpenClaw's saved transcript.

Expected Behavior

OpenClaw should either:

  1. Continue consuming the SSE stream until the connection closes (not stop at finish_reason: stop), or
  2. Fall back to llama.cpp's timings / token fields in the response when the usage chunk was missed

Reproduction

  • Provider: studio-llamacpp with api: openai-completions
  • Any llama.cpp server with streaming enabled
  • Model with reasoning: true (triggers reasoning-budget: activated)
  • Run any agent turn → check saved transcript usage fields → all zeros
  • curl the same endpoint directly → final SSE chunk contains correct usage

Impact

  • /status shows ?/? context instead of actual token counts
  • Compaction/safeguard logic can't accurately track context usage
  • Agents running on local llama.cpp backends effectively have no token visibility

Metadata

Metadata

Assignees

Labels

P2Normal backlog priority with limited blast radius.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:data-lossCan lose, corrupt, or silently drop user/session/config data.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

Type

No type

Fields

Priority

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions