Repository navigation
Commit 4f41a8e
authored
feat(usage): answer today's cost from the CLI in one command (#2396)
* devlog: plan CLI usage cost query
A user asking "how much did Grok cost me today?" could not answer it with
ocx usage. The proxy prices every request at read time, then the CLI discards
the breakdown before printing: summaryLines() stops at depth 1 and renders
models/providers/accounts as "N item(s)". There is also no today window, day
rows carry no cost field at all, and --surface selects the client rather than
the upstream provider, so --surface grok answers a different question than the
one being asked.
Roadmap unit with diff-level decade docs (data -> api -> cli -> verify) plus
the A-phase audit fold-back. Two independent reviews found seven blockers,
including one that would have poisoned the shared usage cache: the cache key
is range:surface, so a filtered summary written under it would serve filtered
totals to the next unfiltered caller, GUI included.
* feat(usage): add a today window and day-level cost attribution
The proxy prices every request as it reads the log, but day rows carried only
requests and tokens, so per-day cost did not exist anywhere - not even in
--json. Anyone asking what today cost had to re-derive prices the proxy had
already computed.
buildDayGrid now prices through the same seam buildModels uses: combo requests
are priced per attempt and each attempt's cost is attributed to its own model,
everything else to the entry's model. Pricing it a second way would have made
days[] disagree with models[] for exactly the combo traffic where the
disagreement is hardest to spot.
Three details worth naming:
- rangeWindow has no exhaustive switch; its final return is the "all" window.
A today member that failed to reach its own branch would compile clean and
silently report all-time history, so today is handled first and the test
asserts since is bounded rather than only checking the day count.
- The day overflow row now sums cost. Past 256 distinct provider/model pairs
the tail collapses into one "other" row, and an unsummed aggregation looks
correct in every breakdown below the cap. The regression test puts the only
priced model in the tail and was driven red before it passed.
- 1d normalises to today at the parser instead of becoming a second union
member, which would have needed its own cache slot and grid arm for nothing.
USAGE_RANGES and USAGE_SURFACES are exported so consumers stop re-declaring
the members in their own literals.
* feat(usage): filter /api/usage by provider and model
Narrowing usage to one provider previously meant fetching the whole window and
filtering client-side. --surface looked like it should help and does not: it
selects the client (codex/claude/grok), not the upstream provider, so
--surface grok answers a different question than the one being asked.
The filter is a projection over a finished summary, never a parameter to
summarizeUsage. That is not a style preference. The cache key is range:surface
and the warm loop writes every key on a miss, so a filtered summary that
reached the producer would be stored under the unfiltered key and the next
caller - the dashboard included - would be served one provider's totals as the
whole window. Keeping the filter outside the producer makes that
unrepresentable rather than merely discouraged.
The regression test for it was verified by falsification: projecting into the
cached value makes it fail, and it passes again when reverted.
Totals are recomputed from retained rows, which is exact for cost (combo cost
is attributed per attempt, so it partitions) but can overcount requests when a
combo request participates in several models. comboOverlap reports when that is
possible instead of leaving it silent. accounts is emptied under a filter
because account rows cannot be honestly re-partitioned by provider, and showing
whole-window account totals beside filtered model totals invites the wrong
reading.
The warm loop now iterates USAGE_RANGES/USAGE_SURFACES instead of its own
literals. A subset literal type-checks happily, which is why today would have
been silently unwarmed and never invalidated with its siblings. The cache-count
assertions in settings-stream-mode are derived from the same constants for the
same reason.
* feat(cli): print the usage cost breakdown instead of an item count
ocx usage rendered its payload through summaryLines(), a generic depth-1
flattener shared with storage/memory/debug/claude-inbound/injection. It stops
at depth 1 and renders any array as "N item(s)", so models, providers and
accounts printed as a row count and every per-entity cost the server had
already computed was discarded. The only cost a user could see was the
whole-window total across every provider, which is not the number anyone is
asking for.
usage now has its own renderer rather than a deeper shared one, because
deepening summaryLines() would change five unrelated commands. It follows the
existing house style: dynamic padEnd columns like formatAccountTable, plain
text, no ANSI.
Two wording decisions are load-bearing. A zero total is ambiguous between "no
spend" and "no price row matched", so unpriced and unmetered counts are shown
separately. And most traffic through this proxy is subscription or OAuth-plan
based where no per-request charge exists at all, so the disclaimer is not
decoration - a bare dollar figure would be read as a bill. Both borrow the
dashboard's existing wording so the two surfaces agree.
--provider and --model are registered in all three places the CLI needs
(the observe USAGE constant, the registry entry, and the help banner); the
banner line was also stale, omitting --surface and --json.
Live evidence in devlog 031: the question that started this unit now answers
in one command.
* docs: document the usage today window and provider filter
English carries the full explanation, including the distinction that keeps
catching people out: --surface selects the calling client, --provider selects
the upstream target serving the request. Locales get the flag list, which is
code-shaped and locale-independent, so they cannot contradict the English
source while awaiting translation.
Also states why a zero cost is not automatically free: requests with no
matching price row are reported as unpriced/unmetered rather than folded in.
* fix(usage): re-summarise filtered windows instead of projecting rows
Review found three defects that shared one cause: the projection operated on
breakdown rows, which have already lost the identity it needed.
- A provider or model living only past MAX_USAGE_MODEL_BREAKDOWN_ROWS is
collapsed into a synthetic "other" row, so filtering for it reported
matched:false and zero cost despite real usage.
- A provider row is a whole-provider aggregate. Under a model filter,
providers[] kept the provider's other models while models[] and the totals
excluded them - one response contradicting itself.
- A model row carries a single optional cost, so priced/unpriced could only be
guessed per model rather than counted per request, and unmetered was always
reported as zero.
Filtering now re-summarises from the entries the summary was built from. The
entries are already in hand on every path that filters, so the honest
computation is also the simpler one. Filtered requests skip the cache, since a
cached summary carries no entries and serving one would mean projecting over
collapsed rows again; the unfiltered cache stays warm either way.
Also: build the human report only when it will be printed. Arguments are
evaluated before the call, so passing it inline ran the renderer during --json
and let its assumptions touch a path meant to bypass it. And 1d now appears in
every synopsis that already listed today - the alias was accepted but
undocumented in all eight of them.
* fix(usage): filter combos by attempt, and scope matched to the window
Two more review findings, both reproduced before fixing.
Keeping a whole combo entry because one of its attempts matched dragged the
other attempts into the filtered totals. A two-attempt combo filtered to its
cheap model reported ~$0.368 instead of ~$0.008 - the expensive model's spend,
attributed to a model the user had explicitly filtered out. The predicate now
applies at attribution level and the entry is rewritten down to its matching
attempts, so filtered numbers mean what the flag says. Combo requests are still
counted once per participating model; that overlap is documented and is what
comboOverlap reports.
filter.matched was computed from entries scanned before the range and surface
predicates ran, so a match from an earlier day set matched:true for
--range today while the summary showed zero requests. Since the CLI uses that
flag to choose between a table and "no usage recorded", the message and the
numbers could disagree. It now derives from the projected summary.
Also adds --model to the top-level help synopsis, which listed only --provider.
* fix(usage): spend each day-attribution cost once
dayAttributionCosts keys by provider/model and holds the SUM of every attempt that
shares a key, but usageAttributions yields one entry per attempt. A retry onto the same
model therefore added that pair's total once per attempt, doubling days[].estimatedCostUsd
against summary.estimatedCostUsd. buildModels never had this bug because it adds each
attempt once.
Delete the key on read so the first attribution carries the group's cost and its siblings
get zero. The regression test uses two attempts on the SAME model, which is the case every
existing combo test misses -- they all use two different models.1 parent 46d4150 commit 4f41a8e
26 files changed
Lines changed: 1863 additions & 37 deletions
File tree
- devlog/_plan/260822_cli_usage_cost_query
- docs-site/src/content/docs
- fr/reference/cli
- ja/reference/cli
- ko/reference/cli
- reference/cli
- ru/reference/cli
- tr/reference/cli
- zh-cn/reference/cli
- zh-tw/reference/cli
- src
- cli
- server/management
- usage
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
0 commit comments