Repository navigation
Price free Codex Auto-review requests at $0 - #4453
Conversation
Auto-review checks stopped drawing from ChatGPT plan usage on 2026-10-06. Price codex-auto-review requests at exactly $0 when the request is at or after the cutoff and the rollout's leaf session_meta has creator_account_id. API-key sessions, earlier requests, and unreconciled rows stay unpriced.
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex review: needs maintainer review before merge. What this changesThis PR prices eligible Codex Auto-review requests at $0 while preserving tokens, saved monetary amounts, and explicit custom rates. Example: An October 8 ChatGPT-created rollout contains two Auto-review requests alongside $0.2350 of other usage.
Review scores
ProductKind: Bug fix · Worth it: Yes · Fix scope: Complete Merge readiness✅ Ready for maintainer review Keep this PR open: it addresses a current cost-reporting bug, and the maintainer’s new commits resolve the previous pricing-policy decision. No actionable introduced defect remains. Priority: P2 Before mergeNone. FindingsNone. Agent review detailsHow this fits togetherThe shared local-cost scanner reads Codex rollout metadata and token records, retains accounting in SQLite, and produces reports consumed by the CLI and Usage & Spend dashboard. flowchart TD
A[Codex rollout files] --> B[Shared scanner]
B --> C[SQLite cost cache]
C --> D[Report time pricing]
E[Custom pricing overlay] --> D
D --> F[Cost CLI]
D --> G[Usage and Spend dashboard]
Technical reviewBest possible solution: Land the scoped shared-scanner fix with the accepted pricing assumptions and existing custom-rate and cache-preservation contracts. Do we have a high-confidence way to reproduce the issue? The affected-release report, supplied before/after CLI output, and main’s unresolved-price path establish the lower-bound behavior; this read-only review ran no target code. Is this the best way to solve the issue? The current implementation uses the existing request-pricing owner, retains uncertain historical rows, and prevents uncovered free groups from falling back to list prices. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against e92d0ed0ff03. Provenance checked
TestingProof path: shipped entry point. SecurityNone. EvidenceWhat I checked:
Likely related people:
Review metrics
Root-cause clusterRelationship: Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. LabelsLabel changes:
Label justifications:
Rating scale6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior. WorkflowClawSweeper edits this one comment on every review. Comment HistoryReview history (3 earlier review cycles)Reviewed October 12, 2026, 12:21 AM ET / 04:21 UTC (Revision 4). |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6ee04fb846
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A model/day group whose rows don't account for its billed tokens fell back to the aggregate list price when it held a free Auto-review request, and its hourly slices list-priced that request. Free requests now block the aggregate fallback and stay unpriced outside honored groups; a free request without tokens is still an exact $0.
Custom-pricing overlays keep precedence over dated pricing changes, and the 2026-10-06 Auto-review waiver is one. An eligible request whose model has a custom rate is now priced at that rate, like any custom-priced request; models.dev and bundled rates still never apply to it.
|
Custom rates. My decision as the author is to keep explicit custom rates. If Shipped CLI proof.
Text output goes from env -i HOME="$H" CFFIXED_USER_HOME="$H" CODEX_HOME="$H/.codex" CODEXBAR_CONFIG="$H/.codexbar/config.json" \
PATH=/usr/bin:/bin SHELL=/bin/sh CODEXBAR_DISABLE_KEYCHAIN_ACCESS=1 CODEXBAR_SUPPRESS_TEST_KEYCHAIN_ACCESS=1 \
"$CLI" cost --provider codex --days 30 --format json --pretty # and --format textEach run used a fresh temporary |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 636af1f1f1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| CostUsagePricing.normalizeCodexModel(row.model) == self.autoReviewModel | ||
| else { return false } | ||
| let pricedModel = CostUsageScanner.codexPricedModel(for: row, priorityTurns: priorityTurns) | ||
| return customPricing.rates(providerID: CostUsagePricing.codexModelsDevProviderID, model: pricedModel) == nil |
There was a problem hiding this comment.
Apply custom rates to retained Auto-review rows
When an otherwise eligible Auto-review row carries unpricedTokens from source recovery or a parser-revision migration, finding a custom rate here makes isIncluded false but does not remove that retained marker. codexRowCostBreakdown then treats the row as isMarkedUnpriced and skips codexResolvedCostUSD, so the explicit codex-auto-review overlay added by this change never applies and the request remains unknown; handle this custom override before the retained-marker guard or clear the marker on the report-time copy.
Useful? React with 👍 / 👎.
Reuse exact request pricing for eligible Auto-review without losing tokens. Keep leaf creator metadata, bounded parser migration, and focused regression coverage; document the accepted cutoff and creation-time auth limitation. Remove redundant metadata initialization and timestamp merge helpers. Fixes steipete#4452 Refs steipete#4453 Co-authored-by: Gabriel Rojas <[email protected]>
Incorporate the contributor's custom-rate precedence and uncovered-group corrections without restoring the separate included-usage subsystem. Resolve free requests through the shared pricing path and keep hourly coverage incomplete when the daily group cannot price its request rows. Retain explicit user rates, saved amounts, and the existing token totals. Verified 290 focused tests and synthetic cost CLI output. Production stays below the base line count; the complete suite is running separately. Fixes steipete#4452 Refs steipete#4453 Co-authored-by: Gabriel Rojas <[email protected]>
|
Thanks @gabrielrojasc! Maintainer fixes are pushed to your branch at 5f2af75, preserving your commits, including the custom-rate and uncovered-group corrections. The implementation now reuses the shared pricing resolver with a report-only eligibility flag; production is net −2 lines against main, and the test addition is 116 lines. The credited changelog entry is under 0.74.2 Unreleased, and the docs state the accepted cutoff assumption and creation-time auth limitation. Merged into main as db753e8. The regression fails on main and passes here; all 290 focused tests and all 1,655 full-suite selections pass, as do make check and autoreview. The synthetic CLI reports $0 with unchanged tokens and no unpriced requests. CI passed on this exact head: https://github.com/steipete/CodexBar/actions/runs/38202794457. Fixes #4452. |
Requests for codex-auto-review in a rollout whose session_meta names a ChatGPT creator_account_id, at or after 2026-10-06T07:00Z, are keyed apart as included usage. They keep their tokens in totals, price at exactly zero, and are not listed as unpriced. Explicit custom prices still apply. Other requests are unchanged. Bump the Codex cache schema to 7 so cached aggregates are rebuilt. Upstream steipete#4452, steipete#4453.
Fixes #4452.
Summary
A
codex-auto-reviewrequest is now priced at an exact $0, instead of left unpriced, when both hold:2026-10-06T07:00:00Z(midnight US Pacific on the day OpenAI made Auto-review free for ChatGPT sign-in);session_metahas a non-emptycreator_account_id. Codex sets that field only for ChatGPT-backed auth, so API-key sessions don't have it.Its tokens stay in day, model, session, and project totals. Auto-review stops turning the Usage & Spend Codex cost into a lower bound. An explicit custom-pricing rate for
codex-auto-reviewstill wins over $0.creator_account_id, a blank one, no request timestamp, a cache entry written before the flag existed, or model/day rows that don't account for the billed tokens. Auto-review before the cutoff and every other unpriced model are unchanged.session_metasets the flag, matching Codex, which never updatescreator_account_idon resume. Asession_metaappended later can't turn an API-key session into a free one, and incremental and full parses agree.docs/model-pricing.mdgives custom overlays precedence over dated pricing changes, and the waiver is one. With acodex-auto-review(oropenai/codex-auto-review) overlay entry, the request is priced like any custom-priced request, including the day's aggregate estimate when its rows don't cover the day. Without one, models.dev and bundled prices never apply. A retained authoritative amount (knownCostNanos) still wins.unmeteredRequestCount > 0as "tokens missing from totals", because missing-parent forks use it that way. Free auto-review tokens are in the totals, so a priced $0 keeps those views exact. Fork accounting is unchanged.isIncludedInPlanexists only on report-time copies. Stored rows don't change.6eedcbb17b6035b4. Thef406dae284de4a55store is adopted without a rebuild. Until the catch-up finishes, the previously retained report (with≥) can still show, which only over-reports.Known limitations
CodexLocalProjectUsageIndexer,CodexWorkspaceUsageSidecar) rebuild session metadata from sidecar columns that don't carry the flag. They still count auto-review as unpriced. Fixing that needs a sidecar schema change, so it's out of scope here.coverageCounts, while a partial day counts requests (the convention from Keep priced Codex subtotals when some requests are unpriced #4279). On real history, the "priced" count drops from 9,510 to 7,065 because Oct 7 to 11 are now fully priced. Dollar and token totals don't change.Evidence
Shipped CLI before and after, synthetic history. Three synthetic rollouts, with invented ids and token counts and no personal history:
creator_account_id) with onegpt-5.4request and twocodex-auto-reviewrequests;creator_account_id) with onecodex-auto-reviewrequest;codex-auto-reviewrequest."Before" is the shipped
CodexBarCLI0.74.0 (/Applications/CodexBar.app/Contents/Helpers/CodexBarCLI, parser hashf406dae284de4a55). "After" isswift build --product CodexBarCLIat636af1f1f(parser hash6eedcbb17b6035b4). Both rancost --provider codex --days 30in JSON and text. The "custom" runs add{"codex-auto-review":{"input":1.25,"output":10,"cacheRead":0.125}}ascustom-pricing.jsonin the temporary HOME.Text output changes from
Partial estimate: 4 recorded requests had no price.toPartial estimate: 2 recorded requests had no price.. The two left are the API-key and pre-cutoff requests. Oct 8 becomes fully priced, so it counts as one priced unit. With a custom rate, results match the shipped CLI.Each run used a fresh HOME with
HOME,CFFIXED_USER_HOME,CODEX_HOME, andCODEXBAR_CONFIGpointed into it, plusCODEXBAR_DISABLE_KEYCHAIN_ACCESS=1, as inScripts/test_dashboard_private_accounts.py.sandbox-execdenied reads and writes to the live CodexBar Application Support and Caches directories,~/.codex, and~/.codexbar. Nothing under the live directories changed during the runs, and each CLI wrote its cost cache and models.dev catalog under the temporary HOME.Reproduce
Write the fixtures with this script (
python3 -I gen_fixture.py "$HOME_UNDER_TEST/.codex"):Then run each CLI against its own fresh HOME:
Real history, before and after, from an earlier head. This run predates rounds 1 and 2 (the uncovered-group fix and the custom-rate precedence) and was not repeated. I ran the in-process scanner over a real
~/.codexhistory, read-only. Each run used its own temporary cache root and suppressed Keychain access, so it didn't touch the app's live cache. Builds came frommain(444821be5) and from this branch at6ee04fb84, run back to back for month to date. The third run upgraded a copy of themainstore in place.f406dae284de4a556ee04fb84, fresh cache81dc6c7f80a36d946ee04fb84, adoptedmainstore81dc6c7f80a36d94The 731 requests still unpriced are auto-review requests from Oct 1 to 6, before the cutoff. Days with unpriced requests go from Oct 1 to 11 down to Oct 1 to 6.
New tests. Each was written before the fix, and the suites fail with the fix disabled.
CostUsageCodexAutoReviewTests, after the cutoff in a ChatGPT session:CostUsageCodexAutoReviewTests, what stays unpriced:CostUsageCodexAutoReviewTests, pricing details:openai/key) prices the request in the day, model breakdown, hourly, and quota slices, and coverage counts it as priced;knownCostNanoswins;eventIndexkeeps the day unpriced and no slice claims $0.CostUsageCodexAutoReviewTests, caching and parsing:session_metawith a creator gives the same result from incremental and full parses;SpendDashboardCodexAutoReviewTestsbuilds its report through the scanner. It checks that the dashboard shows no partial cost and no≥, and that CLI JSON coverage has zero unpriced and unmetered requests.Commands run:
make teststops atAmpUsageParserTests"parses current amp subscription line and keeps credits detail". That test fails the same way onmain(444821be5) and is unrelated to this change.Merge Danger
Door: two-way
Stored rows don't change. The flag lives in cached session metadata, and the classification happens at report time. Reverting restores the previous rule after the next parser-hash reparse.
Blast Radius: Codex costs
On days with
codex-auto-reviewrequests after 2026-10-06 in ChatGPT sessions, those requests move from unpriced to priced at $0. A day with no other unpriced requests stops reading as a partial estimate, and its coverage count follows the fully priced day convention. Dollar totals and token totals don't change. Users with a custom rate forcodex-auto-reviewkeep it. API-key sessions, earlier days, other models, fork accounting, and other providers are unchanged. Every user does a one-time bounded reparse of files in the scan window.