|
| 1 | +# Codex Speed Inference Plan |
| 2 | + |
| 3 | +## Context |
| 4 | + |
| 5 | +Codex logs do not currently record an explicit fast-mode flag per turn. The |
| 6 | +current `--speed auto` behavior reads the current `~/.codex/config.toml` |
| 7 | +`service_tier` and applies one speed choice to all historical Codex usage. That |
| 8 | +means historical cost can change when the current Codex config changes. |
| 9 | + |
| 10 | +Recent log checks showed that raw TPS signals exist, but the denominator matters. |
| 11 | +Simple time between `token_count` events is too noisy because replayed history, |
| 12 | +sub-agent activity, tool calls, and tool output can distort the interval. |
| 13 | + |
| 14 | +Ground-truth notes from local sessions: |
| 15 | + |
| 16 | +- Weekdays in the previous week were mostly fast mode. |
| 17 | +- `kindle-alpha` should generally be treated as fast-mode usage. |
| 18 | +- For today's two sessions, the smaller session was fast and the larger session |
| 19 | + was not fast, so raw turn-level `output_tokens / duration_ms` can classify in |
| 20 | + the wrong direction. |
| 21 | + |
| 22 | +## Goals |
| 23 | + |
| 24 | +- Preserve existing `--speed auto` behavior for compatibility. |
| 25 | +- Add an opt-in inference mode instead of silently changing auto pricing. |
| 26 | +- Infer speed per turn, not per session or per report bucket. |
| 27 | +- Keep standard and fast usage in separate cost buckets so one multiplier is not |
| 28 | + applied to a mixed report. |
| 29 | +- Avoid overcharging uncertain turns. |
| 30 | + |
| 31 | +## Proposed CLI Behavior |
| 32 | + |
| 33 | +Add a new Codex speed option: |
| 34 | + |
| 35 | +```sh |
| 36 | +ccusage codex daily --speed infer |
| 37 | +ccusage codex monthly --speed infer |
| 38 | +ccusage codex session --speed infer |
| 39 | +``` |
| 40 | + |
| 41 | +Speed modes: |
| 42 | + |
| 43 | +- `--speed auto`: keep current behavior. Read Codex config and apply the resolved |
| 44 | + speed globally. |
| 45 | +- `--speed standard`: force all Codex usage to standard pricing. |
| 46 | +- `--speed fast`: force all Codex usage to fast pricing. |
| 47 | +- `--speed infer`: infer fast/standard/unknown per turn from log timing. |
| 48 | + |
| 49 | +## Inference Strategy |
| 50 | + |
| 51 | +Use `--speed infer` only. |
| 52 | + |
| 53 | +Do not use raw time between `token_count` checkpoints as the primary metric. |
| 54 | +It is useful for diagnostics, but too noisy for pricing. |
| 55 | + |
| 56 | +Prefer a model-output interval: |
| 57 | + |
| 58 | +1. Track turn boundaries from `task_started`, `user_message`, and `turn_context`. |
| 59 | +2. Track model-output events such as `response_item` with `message` or |
| 60 | + `reasoning` payloads. |
| 61 | +3. Exclude tool call and tool output time from the denominator where possible. |
| 62 | +4. Compute TPS from output tokens over the model-output interval. |
| 63 | +5. Mark tiny, short, replayed, or structurally ambiguous samples as `unknown`. |
| 64 | + |
| 65 | +Initial buckets: |
| 66 | + |
| 67 | +- `fast`: strong fast signal, or `kindle-alpha` fast prior. |
| 68 | +- `standard`: strong standard signal. |
| 69 | +- `unknown`: borderline, too few valid steps, too few output tokens, missing |
| 70 | + timing events, or replay-like logs. |
| 71 | + |
| 72 | +Initial fallback: |
| 73 | + |
| 74 | +- Treat `unknown` as standard for cost calculation. |
| 75 | + |
| 76 | +This is conservative and avoids bringing back the core bug where current config |
| 77 | +changes historical cost. |
| 78 | + |
| 79 | +## Model Prior |
| 80 | + |
| 81 | +For `--speed infer`, treat `kindle-alpha` as fast by default. Implement this in a |
| 82 | +small helper so it is easy to adjust or override later. |
| 83 | + |
| 84 | +Do not apply this prior to `--speed auto`, `--speed standard`, or `--speed fast`. |
| 85 | + |
| 86 | +## Data Model Changes |
| 87 | + |
| 88 | +The current `CodexModelUsage` stores only one token aggregate per model. That is |
| 89 | +not enough once fast and standard usage can be mixed in the same day, month, or |
| 90 | +session. |
| 91 | + |
| 92 | +Add speed-aware buckets, for example: |
| 93 | + |
| 94 | +```text |
| 95 | +CodexModelUsage |
| 96 | + standard |
| 97 | + fast |
| 98 | + unknown |
| 99 | +``` |
| 100 | + |
| 101 | +Each bucket should carry the same token fields currently stored on |
| 102 | +`CodexModelUsage`: |
| 103 | + |
| 104 | +- input tokens |
| 105 | +- cached input tokens |
| 106 | +- output tokens |
| 107 | +- reasoning output tokens |
| 108 | +- total tokens |
| 109 | + |
| 110 | +Cost calculation should price each bucket separately: |
| 111 | + |
| 112 | +- standard bucket: normal pricing |
| 113 | +- fast bucket: pricing multiplied by `fastMultiplier` |
| 114 | +- unknown bucket: standard pricing |
| 115 | + |
| 116 | +## Parser Changes |
| 117 | + |
| 118 | +Extend Codex parsing to collect timing metadata alongside token usage. |
| 119 | + |
| 120 | +Each `CodexTokenUsageEvent` should have enough information for inference, such |
| 121 | +as: |
| 122 | + |
| 123 | +- turn id, if available |
| 124 | +- model-output interval duration |
| 125 | +- output tokens for the inferred interval |
| 126 | +- inference confidence or bucket |
| 127 | + |
| 128 | +If timing data is unavailable, keep the event usable for token accounting and |
| 129 | +mark the speed bucket as `unknown`. |
| 130 | + |
| 131 | +## Tests |
| 132 | + |
| 133 | +Use minimal JSONL fixtures. Do not include real prompt text or assistant output. |
| 134 | + |
| 135 | +Required regression cases: |
| 136 | + |
| 137 | +- `--speed auto` remains config-based and does not use TPS inference. |
| 138 | +- `--speed infer` separates standard and fast buckets within the same report |
| 139 | + period. |
| 140 | +- `kindle-alpha` is fast under `infer`. |
| 141 | +- Unknown or borderline turns fall back to standard pricing. |
| 142 | +- Today's smaller fast-like session is not misclassified as standard because of |
| 143 | + overhead. |
| 144 | +- Today's larger non-fast session is not misclassified as fast by raw |
| 145 | + `output_tokens / duration_ms`. |
| 146 | +- Reports apply `fastMultiplier` only to fast buckets. |
| 147 | +- `ccusage daily` all-agent reports use the same Codex inferred bucket costs as |
| 148 | + `ccusage codex daily`. |
| 149 | + |
| 150 | +## Documentation Updates |
| 151 | + |
| 152 | +If implemented, update: |
| 153 | + |
| 154 | +- root `README.md` |
| 155 | +- `apps/ccusage/README.md` |
| 156 | +- `docs/guide/codex/index.md` |
| 157 | +- relevant all-report examples if they mention Codex speed pricing |
| 158 | +- CLI help snapshots and config schema artifacts |
| 159 | + |
| 160 | +Docs should clearly state: |
| 161 | + |
| 162 | +- Codex logs do not expose an authoritative fast flag. |
| 163 | +- `auto` is config-based. |
| 164 | +- `infer` is timing-based and heuristic. |
| 165 | +- `infer` is conservative for unknown turns. |
| 166 | + |
| 167 | +## Rollout |
| 168 | + |
| 169 | +1. Add `CodexSpeed::Infer` and CLI/config/schema support. |
| 170 | +2. Add failing tests for mixed standard/fast bucket pricing. |
| 171 | +3. Add parser timing extraction with minimal fixtures. |
| 172 | +4. Add inference helper and conservative bucket assignment. |
| 173 | +5. Split Codex aggregation into speed-aware buckets. |
| 174 | +6. Update Codex-focused reports. |
| 175 | +7. Update all-agent reports. |
| 176 | +8. Update docs and generated artifacts. |
| 177 | +9. Run focused Rust tests, then `just fmt` and the relevant repo checks. |
0 commit comments