Skip to content

Fix OpenAI Admin usage pagination#1465

Merged
steipete merged 2 commits into
steipete:mainfrom
rohitjavvadi:fix/openai-admin-pagination
Jun 12, 2026
Merged

Fix OpenAI Admin usage pagination#1465
steipete merged 2 commits into
steipete:mainfrom
rohitjavvadi:fix/openai-admin-pagination

Conversation

@rohitjavvadi

@rohitjavvadi rohitjavvadi commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

OpenAI Admin usage responses can be paginated, but CodexBar was only reading the first page for both costs and completions usage. That could undercount organization usage whenever the API returned has_more: true.

This updates the OpenAI API usage fetcher to follow next_page cursors for both endpoints, preserve the existing range/project/group query parameters on each page request, and fail closed on malformed pagination state instead of silently returning partial totals.

What changed

  • Follow has_more / next_page pagination for OpenAI Admin costs and completions usage.
  • Keep project_ids, group_by, time range, bucket width, and limit query params across page requests.
  • Detect repeated cursors, missing cursors when has_more is true, and excessive pagination.
  • Require top-level data and has_more in Admin usage page responses so malformed pages do not look like empty successful usage.
  • Add regression coverage for multi-page costs/completions, repeated cursors, missing cursors, and malformed page responses.

Live behavior proof

I verified the pagination path against a real OpenAI Admin API response with keys, cursors, project/org identifiers, and model names redacted. The probe used the same Admin costs and completions endpoints with bucket_width=1d and limit=1 to force cursor traversal.

OpenAI Admin live pagination proof, redacted. range=2026-05-14..2026-06-12 bucket_width=1d limit=1

costs:
  pages_fetched=30 final_has_more=false all_next_page_cursors_present_until_final=true
  first_page: buckets=1 results=0 cost_usd=0.000000 requests=0 input_tokens=0 output_tokens=0
  all_pages: buckets=30 results=17 cost_usd=0.000000 requests=0 input_tokens=0 output_tokens=0

completions:
  pages_fetched=30 final_has_more=false all_next_page_cursors_present_until_final=true
  first_page: buckets=1 results=0 cost_usd=0.000000 requests=0 input_tokens=0 output_tokens=0
  all_pages: buckets=30 results=7 cost_usd=0.000000 requests=22 input_tokens=16284907 output_tokens=292570

That shows the old first-page-only behavior would report zero completions usage for this range, while following all pages finds the later usage buckets.

Testing

  • swift test --filter OpenAIAPIUsageFetcherTests --filter OpenAIAPICreditBalanceTests --filter OpenAIAPIMenuCardModelTests
  • make check

I also tried an aggregate swift test --disable-sandbox run after rebasing on latest upstream/main; it hit a timing-sensitive MainThreadHangWatchdogTests failure during the full run, and that same suite passed when rerun directly with swift test --disable-sandbox --filter MainThreadHangWatchdogTests.

@clawsweeper

clawsweeper Bot commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed June 12, 2026, 9:26 AM ET / 13:26 UTC.

Summary
Review failed before ClawSweeper could summarize the requested change.

Reproducibility: unclear. The review failed before ClawSweeper could establish a reproduction path.

Review metrics: none identified.

Merge readiness
Overall: 🌊 off-meta tidepool
Proof: 🌊 off-meta tidepool
Patch quality: 🌊 off-meta tidepool
Result: rating does not apply to this item.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

  • [P1] No close action taken because the review did not complete.

Maintainer options:

  1. Decide the mitigation before merge
    Retry the Codex review after fixing the execution failure.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] Review did not complete, so no work-lane recommendation was made.
Review details

Best possible solution:

Retry the Codex review after fixing the execution failure.

Do we have a high-confidence way to reproduce the issue?

Unclear. The review failed before ClawSweeper could establish a reproduction path.

Is this the best way to solve the issue?

Unclear. Retry the review first so ClawSweeper can evaluate the actual issue and fix direction.

AGENTS.md: unclear because the file could not be read completely.

Codex review notes: model internal, reasoning high; reviewed against 186c6de6678a.

Label changes

Label changes:

  • remove P2: Current review triage priority is none.

Label justifications:

  • rating: 🌊 off-meta tidepool: Overall readiness is 🌊 off-meta tidepool; proof is 🌊 off-meta tidepool and patch quality is 🌊 off-meta tidepool.
Evidence reviewed

What I checked:

  • failure reason: retryable codex transport failure (capacity)
  • codex failure detail: Codex review failed for this PR with exit 1.
  • codex stderr: ests = await transport.requests()\n+ let costsRequests = requests.filter { $0.url?.path.contains("/organization/costs") == true }\n+ let completionRequests = requests.filter { $0.url?.path.contains("/usage/completions") == true }\n+\n+ #expect(snapshot.daily.count == 1)\n+ #expect(snapshot.latestDay.costUSD == 4.0)\n+ #expect(snapshot.latestDay.requests == 3)\n+ #expect(snapshot.latestDay.totalTokens == 45)\n+ #expect(costsRequests.count == 2)\n+ #expect(completionRequests.count == 2)\n+ #expect(Self.queryValue("page", in: costsRequests[0]) == nil)\n+ #expect(Self.queryValue("page", in: costsRequests[1]) == "costs_page_2")\n+ #expect(Self.queryValue("page", in: completionRequests[0]) == nil)\n+ #expect(Self.queryValue("page", in: completionRequests[1]) == "completions_page_2")\n+ #expect(requests.allSatisfy { Self.queryValue("project_ids", in: $0) == "proj_abc" })\n+ }\n+\n+ @test\n+ func admin usage rejects repeated pagination cursor() async throws {\n+ let transport = OpenAIAdminUsageRepeatingCursorScript()\n+\n+ await #expect(throws: OpenAIAPIUsageError.parseFailed(\n+ endpoint: "co\n\n[truncated 10076 chars]".
  • codex stdout: No stdout captured.

Likely related people:

  • unknown: Codex failed before it could trace repository history. (role: review did not complete; confidence: low)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. labels Jun 12, 2026
@steipete
steipete force-pushed the fix/openai-admin-pagination branch from a95e657 to d91a5db Compare June 12, 2026 12:37
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jun 12, 2026
@steipete
steipete force-pushed the fix/openai-admin-pagination branch from d91a5db to 3b2e89e Compare June 12, 2026 13:15
@clawsweeper clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 12, 2026
@rohitjavvadi

rohitjavvadi commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Added redacted live Admin API pagination proof to the PR body.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@steipete
steipete merged commit b9a22ef into steipete:main Jun 12, 2026
4 checks passed
@steipete

steipete commented Jun 12, 2026

Copy link
Copy Markdown
Owner

Landed as b9a22ef.

Proof:

Thanks @rohitjavvadi!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

P2 Normal priority bug or improvement with limited blast radius. rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@rohitjavvadi @steipete