Include token usage in model usage summaries by AntSentry · Pull Request #78247 · openclaw/openclaw

AntSentry · 2026-05-06T03:23:44Z

Summary

include per-model token totals in model-usage --mode all text and JSON output when CodexBar model breakdowns expose token fields
support CodexBar-style token fields and OpenAI-style prompt_tokens / completion_tokens / total_tokens
report tokens unavailable when CodexBar only provides per-model cost, avoiding invented token splits from aggregate daily totals

Real behavior proof

behavior: skills/model-usage/scripts/model_usage.py --mode all now prints and serializes per-model token usage for every model in the breakdown when token fields are present, while still listing cost-only models with an explicit unavailable token state.

environment: Local OpenClaw checkout on macOS using Python 3.14 for the direct script smoke check and a Python virtualenv for repo-style skills checks. The input was a local redacted sample CodexBar cost payload with no secrets.

steps:

Created /tmp/model_usage_sample.json with three model breakdown rows: CodexBar-style token fields, OpenAI-style token fields, and a cost-only row.
Ran python3 skills/model-usage/scripts/model_usage.py --provider codex --mode all --input /tmp/model_usage_sample.json.
Ran python3 skills/model-usage/scripts/model_usage.py --provider codex --mode all --format json --input /tmp/model_usage_sample.json.

evidence:

Provider: codex
Models:
- gpt-5.5: $2.00, tokens 150 (in 100, out 50)
- glm-5.1: $1.00, tokens 275 (in 200, out 75)
- legacy: $0.25, tokens unavailable
{"provider": "codex", "mode": "all", "models": [{"model": "gpt-5.5", "totalCostUSD": 2.0, "tokensAvailable": true, "totalTokens": 150, "inputTokens": 100, "outputTokens": 50, "cacheReadTokens": 0, "cacheCreationTokens": 0}, {"model": "glm-5.1", "totalCostUSD": 1.0, "tokensAvailable": true, "totalTokens": 275, "inputTokens": 200, "outputTokens": 75, "cacheReadTokens": 0, "cacheCreationTokens": 0}, {"model": "legacy", "totalCostUSD": 0.25, "tokensAvailable": false, "totalTokens": null, "inputTokens": null, "outputTokens": null, "cacheReadTokens": null, "cacheCreationTokens": null}]}

observedResult: All three models appear in both text and JSON. The two models with token fields include computed token totals and input/output details. The cost-only model remains listed and reports tokens unavailable / tokensAvailable: false, so the command does not invent a per-model token split from aggregate daily totals.

notTested: I did not run against live CodexBar logs because this environment does not have the CodexBar CLI available on PATH; the verified smoke input mirrors the documented modelBreakdowns[] payload shape.

Verification

python3 -m py_compile skills/model-usage/scripts/model_usage.py skills/model-usage/scripts/test_model_usage.py
python3 skills/model-usage/scripts/test_model_usage.py
/tmp/openclaw-skills-venv/bin/python -m ruff check --config skills/pyproject.toml skills/model-usage
/tmp/openclaw-skills-venv/bin/python -m pytest -q -c skills/pyproject.toml skills/model-usage

Notes

bd sync could not run in this checkout because no beads database exists.

clawsweeper · 2026-05-06T03:31:30Z

Codex review: needs changes before merge. Reviewed July 3, 2026, 5:04 PM ET / 21:04 UTC.

Summary
The PR updates the bundled model-usage skill docs, Python helper, and tests so --mode all can include per-model token availability and totals in text and JSON output.

PR surface: Docs +1, Other +159. Total +160 across 3 files.

Reproducibility: yes. for the review findings: source inspection of PR head, current main, and upstream CodexBar source shows both the finite-cost regression path and the totalTokens-only split-nullability problem. The overall PR is a bundled skill enhancement, not a current-main bug report.

Review metrics: 1 noteworthy metric.

All-mode JSON token fields: 6 added. The PR adds machine-readable token availability and split fields to --mode all, so nullability semantics matter before merge.

Stored data model
Persistent data-model change detected: persistent cache schema: skills/model-usage/SKILL.md. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🦞 diamond lobster
Patch quality: 🦐 gold shrimp
Result: needs maintainer review before merge.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Refresh the branch against current main.
Reuse coerce_finite_cost() in token-aware aggregation and preserve unknown split-token fields as null.

Risk before merge

[P1] The PR is currently CONFLICTING/DIRTY, so maintainers cannot trust the final merged behavior until the branch is refreshed and the conflict resolution is reviewed.
[P1] If the token-aware aggregation keeps the PR head's raw native-number cost check, current numeric-string, boolean, and non-finite cost protections can regress.
[P1] The new all-mode JSON token fields can become a downstream contract; missing split-token fields should not be serialized as known-zero when only per-model totalTokens is known.

Maintainer options:

Compose With Current Cost Semantics (recommended)
Refresh the branch, route token-aware cost aggregation through coerce_finite_cost(), and add coverage for numeric-string costs plus totalTokens-only rows.
Define Zero-As-Known Explicitly
Maintainers could intentionally define absent split counters as zero, but that JSON contract should be documented and tested before merge.
Pause Until Branch Refresh
Keep the PR open but do not merge it until the author or a repair pass resolves the dirty branch against current main.

Copy recommended automerge instruction

@clawsweeper automerge

Special instructions:
Refresh the PR branch against current main; keep `coerce_finite_cost()` in the token-aware model aggregation and all cost readers; model token fields should distinguish unavailable split counters from known zero values and emit `null` for absent input/output/cache splits; add regression coverage for numeric-string cost in token aggregation and for totalTokens-only `modelBreakdowns[]` rows; run the focused model-usage Python checks.

Next step before merge

[P2] A focused repair can refresh the dirty branch and reconcile token aggregation with current cost coercion and token nullability semantics.

Security
Cleared: The diff is limited to a local bundled skill script, skill docs, and tests, with no dependency, workflow, package, secret, lockfile, or new external code-execution surface.

Review findings

[P1] Reuse finite cost coercion in model aggregation — skills/model-usage/scripts/model_usage.py:149-150
[P2] Preserve unknown split-token fields — skills/model-usage/scripts/model_usage.py:320-323

Review details

Best possible solution:

Refresh the branch onto current main and land a narrow model-usage skill enhancement that reuses coerce_finite_cost() and emits null for unavailable split-token fields.

Do we have a high-confidence way to reproduce the issue?

Yes for the review findings: source inspection of PR head, current main, and upstream CodexBar source shows both the finite-cost regression path and the totalTokens-only split-nullability problem. The overall PR is a bundled skill enhancement, not a current-main bug report.

Is this the best way to solve the issue?

No, not as submitted. Parsing token data at the modelBreakdowns[] boundary is the right layer, but the implementation must preserve current cost coercion and avoid turning unavailable split-token fields into zero-valued JSON fields.

Full review comments:

[P1] Reuse finite cost coercion in model aggregation — skills/model-usage/scripts/model_usage.py:149-150
Current main routes model costs through coerce_finite_cost(), accepting numeric strings and rejecting booleans/NaN/Infinity. This new aggregation path goes back to native-number checks, and aggregate_costs() now returns through it, so a refreshed merge must call the shared helper before adding cost.
Confidence: 0.92
[P2] Preserve unknown split-token fields — skills/model-usage/scripts/model_usage.py:320-323
Current CodexBar per-model breakdowns expose totalTokens but not input/output/cache splits. When only totalTokens is present, these fields serialize as 0, which tells JSON consumers the split is known-zero rather than unknown; track split availability separately and emit null for absent splits.
Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.92

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 010b61746379.

Label changes

Label justifications:

P2: This is a bounded bundled-skill enhancement with concrete correctness blockers and limited core-runtime blast radius.
merge-risk: 🚨 compatibility: The PR changes all-mode JSON output and model aggregation semantics in a way that can mislead consumers or regress current cost handling if merged as-is.
rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body includes after-fix commands and copied output from the actual script against a redacted CodexBar-shaped JSON sample, which is sufficient for this local CLI skill behavior.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix commands and copied output from the actual script against a redacted CodexBar-shaped JSON sample, which is sufficient for this local CLI skill behavior.

Evidence reviewed

PR surface:

Docs +1, Other +159. Total +160 across 3 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	0	0	0	0
Tests	0	0	0	0
Docs	1	3	2	+1
Config	0	0	0	0
Generated	0	0	0	0
Other	2	172	13	+159
Total	3	175	15	+160

Acceptance criteria:

[P1] PYTHONDONTWRITEBYTECODE=1 python3 -B -m py_compile skills/model-usage/scripts/model_usage.py skills/model-usage/scripts/test_model_usage.py.
[P1] PYTHONDONTWRITEBYTECODE=1 python3 -B skills/model-usage/scripts/test_model_usage.py.
[P1] python3 -m ruff check --config skills/pyproject.toml skills/model-usage.
[P1] python3 -m pytest -q -c skills/pyproject.toml skills/model-usage.

What I checked:

Current main remains cost-only for all-mode JSON: Current build_json_all emits only model and totalCostUSD, so the central token-output behavior is not implemented on main. (skills/model-usage/scripts/model_usage.py:259, 010b61746379)
Latest release is also cost-only: Tag v2026.6.11 contains the same cost-only all-mode text and JSON implementation, so the requested behavior has not shipped. (skills/model-usage/scripts/model_usage.py:259, e085fa1a3ffd)
Current main finite-cost contract: Current main routes model costs through coerce_finite_cost(), accepting numeric strings and rejecting booleans or non-finite values. (skills/model-usage/scripts/model_usage.py:111, 010b61746379)
PR head bypasses finite-cost coercion: The token-aware aggregation path on the PR head adds costs only when they are native int or float, which would regress the merged cost-coercion behavior if preserved during conflict resolution. (skills/model-usage/scripts/model_usage.py:149, d594d8aebf7d)
PR head serializes missing split counters as zero: Once tokensAvailable is true, the PR serializes input/output/cache counters as zero even when only a total-token field was present. (skills/model-usage/scripts/model_usage.py:320, d594d8aebf7d)
CodexBar per-model token contract: Current CodexBar source encodes per-model modelName, cost, and totalTokens, but not per-model input/output/cache split fields. (Sources/CodexBarCLI/CLICostCommand.swift:327)

Likely related people:

Peter Steinberger: git log --all and git show show f0abd619 added the model-usage skill, script, and CodexBar reference that define this bundled skill surface. (role: feature introducer; confidence: high; commits: f0abd619be69; files: skills/model-usage/SKILL.md, skills/model-usage/references/codexbar-cli.md, skills/model-usage/scripts/model_usage.py)
Vincent Koc: c8a62e1c hardened nearby Python skill behavior and PR metadata shows this handle merged the finite-cost coercion repair that this PR must preserve. (role: recent area contributor and merger; confidence: high; commits: c8a62e1cea75, d63a73a1b86f; files: skills/model-usage/scripts/model_usage.py, skills/model-usage/scripts/test_model_usage.py)
coder999999999: They authored the merged finite-cost coercion repair whose behavior is now the compatibility baseline for token-aware aggregation. (role: recent adjacent contributor; confidence: medium; commits: 11bb5719caea, d63a73a1b86f; files: skills/model-usage/scripts/model_usage.py, skills/model-usage/scripts/test_model_usage.py)
Mason Huang: 55f05df7 made a small recent cleanup in the same model-usage script before the later cost-coercion work. (role: recent adjacent contributor; confidence: medium; commits: 55f05df77eac; files: skills/model-usage/scripts/model_usage.py)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

RomneyDa · 2026-05-29T06:32:03Z

Heads up: this PR needs to be updated against current main before the new required Dependency Guard check can pass.

openclaw-barnacle Bot added size: S triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 6, 2026

Include token usage in model usage summaries

d594d8a

AntSentry force-pushed the model-usage-token-usage branch from 58e4bee to d594d8a Compare May 6, 2026 03:25

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 6, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 6, 2026

clawsweeper Bot mentioned this pull request May 9, 2026

model-usage: handle numeric-string costs and ignore non-finite values #37878

Closed

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 14, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 14, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 14, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 14, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 14, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 14, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 15, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026

openclaw-barnacle Bot removed the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label Jun 14, 2026

clawsweeper Bot mentioned this pull request Jun 19, 2026

fix(model-usage): coerce numeric-string costs and ignore non-finite values #87861

Merged

clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Jun 24, 2026

openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 24, 2026

clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jun 30, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Include token usage in model usage summaries#78247

Include token usage in model usage summaries#78247
AntSentry wants to merge 1 commit into
openclaw:mainfrom
AntSentry:model-usage-token-usage

AntSentry commented May 6, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented May 6, 2026 •

edited

Loading

Uh oh!

RomneyDa commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

AntSentry commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Real behavior proof

Verification

Notes

Uh oh!

clawsweeper Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RomneyDa commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AntSentry commented May 6, 2026 •

edited

Loading

clawsweeper Bot commented May 6, 2026 •

edited

Loading