fix(agents): parse prompt_tokens/completion_tokens in CLI usage for llama.cpp compatibility (#77992) by Beandon13 · Pull Request #78085 · openclaw/openclaw

Beandon13 · 2026-05-05T21:50:26Z

Summary

toCliUsage() in cli-output.ts only recognized input_tokens/output_tokens (and camelCase aliases) from CLI runner output. llama.cpp and other OpenAI-compatible local providers return prompt_tokens/completion_tokens instead, which are the standard OpenAI field names.
Without the fallback, usage was silently dropped and context display showed ?/131k for all llama.cpp, Ollama, and similar OpenAI-compatible users.
Fix: add prompt_tokens → fallback for totalInput and completion_tokens → fallback for output in toCliUsage(). Both parseCliJson and parseCliJsonl route through this function, so all CLI output parsing paths are covered.

Closes #77992

Testing

pnpm vitest run src/agents/cli-output.test.ts

Real behavior proof

Behavior: Context display shows ?/131k with llama.cpp after upgrading to 2026.5.4 — field name mismatch causes usage to be silently dropped
Tested via targeted unit test added in this PR that exercises the exact llama.cpp response shape (prompt_tokens, completion_tokens, total_tokens).
What was not tested: live runtime — please apply maintainer proof: override or advise on evidence format.

clawsweeper · 2026-05-05T21:53:26Z

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Keep open: current main and the latest release still have the CLI parser alias gap, but this PR is not merge-ready because it duplicates shared usage-normalization logic and still lacks contributor real behavior proof.

Root-cause cluster
Relationship: canonical
Canonical: #78085
Summary: This PR remains the narrow open candidate for the CLI output parser alias fallback; related merged and open items cover gateway projection or embedded streaming/session persistence rather than this parser seam.

Members:

partial_overlap: [Bug] Context display shows ?/131k with llama.cpp after upgrading to 2026.5.4 — field name mismatch not resolved #77992 - The linked llama.cpp context-display issue was closed by gateway projection work, but the CLI parser subset remains separate from that merged fix.
partial_overlap: fix(gateway): preserve OpenAI usage aliases in chat history #85383 - The merged PR preserves OpenAI-compatible aliases through gateway chat-history projection but does not change the CLI output parser.
superseded: fix: robust token usage normalization for OpenAI-compatible providers #45535 - The older closed-unmerged PR attempted broader usage normalization including CLI-related work, but it is conflicting and no longer a viable canonical landing path.
adjacent_distinct: OpenAI-compatible streaming with llama.cpp saves zero usage (stream closed before final usage chunk) #79897 - That closed issue concerns final streamed usage chunks from OpenAI-compatible llama.cpp transport, not CLI JSON/JSONL alias parsing.
adjacent_distinct: [Bug]: Usage always 0 for custom OpenAI-compatible provider when using embedded runner (--local) #96463 - That open issue concerns custom OpenAI-compatible embedded streaming usage/request compatibility, not the CLI output parser.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Canonical path: Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review.

So I’m closing this here because the remaining work is already tracked in the canonical issue.

Review details

Best possible solution:

Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main routes CLI JSON/JSONL usage payloads through toCliUsage(), which still omits prompt_tokens and completion_tokens. I did not run a live llama.cpp or CLI-backed provider setup in this read-only review.

Is this the best way to solve the issue?

No: the PR fixes the immediate alias pair, but normalizeUsage() already owns provider alias normalization and has broader coverage for cache, timing, reasoning, and zero-value behavior. The best fix is to adapt the CLI parser to that shared helper and keep the regression test.

Security review:

Security review cleared: The diff only changes TypeScript usage parsing and a focused unit test; no security or supply-chain surface is touched.

AGENTS.md: found and applied where relevant.

What I checked:

stale F-rated PR: PR was opened 2026-05-05T21:50:26Z, is older than 60 days, and the latest review rated it F.
proof blocker: real behavior proof is missing and proof tier is F, so this branch is not merge-ready without contributor follow-up.
no human follow-up: live comments and timeline hydrated by apply contain no non-automation activity after the ClawSweeper review.

Likely related people:

vincentkoc: Recent merged commits changed src/agents/cli-output.ts and its tests around JSON stats/cache parsing, the same parser seam this PR touches. (role: recent CLI parser contributor; confidence: high; commits: c75f82448fad, bcd0a492a4e0; files: src/agents/cli-output.ts, src/agents/cli-output.test.ts)
steipete: Authored the merged gateway work that preserved OpenAI-compatible usage aliases for display projection, which partially overlaps the linked llama.cpp report. (role: adjacent usage/display contributor; confidence: high; commits: d9c6c5f600ca; files: src/gateway/chat-display-projection.ts, src/gateway/server-methods/server-methods.test.ts)
Lellansin: Authored merged OpenAI-compatible chat-completions usage work that added shared usage mapping coverage adjacent to this alias normalization path. (role: OpenAI-compatible usage contributor; confidence: medium; commits: 2ccd1839f212; files: src/agents/usage.ts, src/gateway/openai-http.ts, src/gateway/openai-http.test.ts)

Codex review notes: model internal, reasoning high; reviewed against 010b61746379.

byungskers

This is a great compatibility fix for llama.cpp and other OpenAI-compatible local providers. The fallback chain for usage fields is well-structured — prioritizing the modern input/output fields first while gracefully falling back to prompt/completion_tokens. The test case with the detailed comment explaining the issue is especially helpful for future maintainers.

…t for llama.cpp (openclaw#77992) llama.cpp and other OpenAI-compatible local providers return usage as { prompt_tokens, completion_tokens } instead of { input_tokens, output_tokens }. The toCliUsage() function in cli-output.ts only accepted input_tokens / output_tokens (and their camelCase aliases), so llama.cpp usage was silently dropped and context display showed "?/131k" for all llama.cpp users. Add prompt_tokens and completion_tokens as fallback keys for totalInput and output respectively in toCliUsage(). Both parseCliJson and parseCliJsonl go through this function, so the fix covers all CLI output parsing paths. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

openclaw-barnacle · 2026-05-31T05:03:44Z

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

clawsweeper · 2026-07-19T10:32:20Z

@Beandon13 thanks for the PR. ClawSweeper is still waiting on real behavior proof before this can move forward.

Useful proof can be a screenshot, short video, terminal output, copied live output, linked artifact, or redacted logs that show the changed behavior after the fix. Please redact private tokens, phone numbers, private endpoints, customer data, and anything else sensitive.

Once proof is added to the PR body or a comment, ClawSweeper or a maintainer can re-check it.

clawsweeper · 2026-07-19T19:40:31Z

ClawSweeper status: review started.

I am starting a fresh review of this pull request: fix(agents): parse prompt_tokens/completion_tokens in CLI usage for llama.cpp compatibility (#77992) This is item 1/1 in the current shard. Shard 8/22.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

openclaw-barnacle Bot added agents Agent runtime and tooling size: XS triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 5, 2026

byungskers reviewed May 5, 2026

View reviewed changes

Beandon13 force-pushed the fix/openclaw-77992-llamacpp-usage-tokens branch from 1eae9bc to 383946f Compare May 6, 2026 12:46

This was referenced May 10, 2026

fix: robust token usage normalization for OpenAI-compatible providers #45535

Closed

[Bug] Context display shows ?/131k with llama.cpp after upgrading to 2026.5.4 — field name mismatch not resolved #77992

Closed

clawsweeper Bot mentioned this pull request May 22, 2026

[Bug]: token counting broken with local LLM, only user prompts counted #85165

Closed

openclaw-barnacle Bot added the stale Marked as stale due to inactivity label May 31, 2026

clawsweeper Bot added the rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. label May 31, 2026

barnacle-openclaw Bot removed the stale Marked as stale due to inactivity label May 31, 2026

clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jun 15, 2026

clawsweeper Bot added the merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. label Jun 28, 2026

openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 28, 2026

clawsweeper Bot mentioned this pull request Jun 28, 2026

OpenAI-compatible streaming with llama.cpp saves zero usage (stream closed before final usage chunk) #79897

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(agents): parse prompt_tokens/completion_tokens in CLI usage for llama.cpp compatibility (#77992)#78085

fix(agents): parse prompt_tokens/completion_tokens in CLI usage for llama.cpp compatibility (#77992)#78085
Beandon13 wants to merge 1 commit into
openclaw:mainfrom
Beandon13:fix/openclaw-77992-llamacpp-usage-tokens

Beandon13 commented May 5, 2026

Uh oh!

clawsweeper Bot commented May 5, 2026 •

edited

Loading

Uh oh!

byungskers left a comment

Uh oh!

openclaw-barnacle Bot commented May 31, 2026

Uh oh!

clawsweeper Bot commented Jul 19, 2026

Uh oh!

clawsweeper Bot commented Jul 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

Beandon13 commented May 5, 2026

Summary

Testing

Real behavior proof

Uh oh!

clawsweeper Bot commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

byungskers left a comment

Choose a reason for hiding this comment

Uh oh!

openclaw-barnacle Bot commented May 31, 2026

Uh oh!

clawsweeper Bot commented Jul 19, 2026

Uh oh!

clawsweeper Bot commented Jul 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented May 5, 2026 •

edited

Loading