Skip to content

fix(oauth): bound github-copilot OAuth response reads at 16 MiB#97499

Merged
vincentkoc merged 2 commits into
openclaw:mainfrom
wangmiao0668000666:fix/copilot-oauth-bound
Jun 28, 2026
Merged

fix(oauth): bound github-copilot OAuth response reads at 16 MiB#97499
vincentkoc merged 2 commits into
openclaw:mainfrom
wangmiao0668000666:fix/copilot-oauth-bound

Conversation

@wangmiao0668000666

@wangmiao0668000666 wangmiao0668000666 commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

src/llm/utils/oauth/github-copilot.ts:178-190 (fetchJson) reads the OAuth response body with await response.text() and response.json(), both of which buffer the entire payload into memory before the runtime can decide to stop. A hostile or buggy GitHub OAuth endpoint (or enterprise proxy) can return 200 OK with a body that grows unbounded, and the runtime will buffer all of it before failing. The existing fetchResponse timeout only caps wall-clock time, not bytes — a slow trickle that exceeds memory still OOMs.

The OAuth surface here is untrusted: <domain>/login/device/code, <domain>/login/oauth/access_token, api.<domain>/copilot_internal/v2/token, and api.individual.githubcopilot.com/models are all external, so a DoS-shaped response is in-scope as a runtime hardening fix.

Why This Change Was Made

src/agents/provider-http-errors.ts already exposes the canonical bounded readers (readProviderTextResponse, readProviderJsonResponse) capped at the shared 16 MiB cap. The sibling reference already in main is src/agents/chutes-oauth.ts:9 — same import shape, same helper, three call sites with labels "Chutes userinfo" / "Chutes token exchange" / "Chutes token refresh". This PR mirrors that pattern for github-copilot OAuth, wiring the existing 4 fetchJson call sites (each already passes a distinct operation string) through the shared helper. No new abstraction, no SDK promotion, no config change — pure per-surface application of an existing helper.

User Impact

  • Existing successful GitHub Copilot OAuth flows keep working — the only new failure mode is GitHub Copilot <op>: JSON response exceeds 16777216 bytes when an OAuth endpoint streams >16 MiB, which is the desired safety behavior.
  • Users behind enterprise proxies returning oversized error bodies see a labeled, capped error rather than an OOM.
  • Error format on non-200 responses (${status} ${statusText}: ${text}) is preserved byte-for-byte.

Changes

  • src/llm/utils/oauth/github-copilot.ts:7-9 — import readProviderJsonResponse and readProviderTextResponse from the shared ../../../agents/provider-http-errors.js (sibling pattern matches src/agents/chutes-oauth.ts:9).
  • src/llm/utils/oauth/github-copilot.ts:182-197fetchJson swaps await response.text() / response.json() for the bounded readers with a per-call label "GitHub Copilot <operation>". Default cap is the shared 16 MiB (PROVIDER_TEXT_RESPONSE_MAX_BYTES / PROVIDER_JSON_RESPONSE_MAX_BYTES from src/agents/provider-http-errors.ts:14-17).
  • src/llm/utils/oauth/github-copilot.test.ts:185-265 — 3 new inline tests in a new describe("GitHub Copilot OAuth bounded reads") block: (1) oversized streaming body triggers overflow with the labeled error and the standard 16 MiB cap, (2) normal-size OAuth JSON parses through, (3) happy-path round-trip through refreshGitHubCopilotToken returns the access token with expires as a number. Tests use real ReadableStream + chunked vi.stubGlobal("fetch"), mirroring the inline pattern from src/agents/provider-transport-fetch.test.ts (Alix-007 fix(googlechat): replace unbounded response.json() with readProviderJsonResponse #96772).

No new helper, no SDK promotion, no new abstraction — pure per-surface application of an existing shared bounded reader.

The four call sites of fetchJson (lines 204, 298, 371, 450) need no edits: each already passes a distinct operation string ("device code request", "device token request", "token refresh request", "model list request"), so the labels "GitHub Copilot <operation>" stay distinct without further changes.

Real behavior proof

Real Node http server on 127.0.0.1, real refreshGitHubCopilotToken wrapper, globalThis.fetch rewritten to point https://api.github.com/... at the local server (production code untouched). Proof script is /tmp/g1_real_server_proof.mts, run once, not committed (per checklist #23).

  • Behavior addressed: A 17 MiB OAuth response body triggers the 16 MiB cap on readProviderJsonResponse instead of buffering the full payload.
  • Real environment tested: Linux Node v22.22.0, real http.createServer listening on ephemeral 127.0.0.1 port, real globalThis.fetch wrapper routing only the GitHub OAuth hostnames to localhost, real production refreshGitHubCopilotToken invoked.
  • Exact steps or command run after this patch:
    node --import tsx /tmp/g1_real_server_proof.mts
  • Evidence after fix (3/3 PASS, captured from real terminal):
    PASS  negative-control: serverGotRequest=true bytesSent=17825792 cap=16777216 msg="GitHub Copilot token refresh request: JSON response exceeds 16777216 bytes"
    PASS  happy-path:        access=copilot-token-XYZ expires=1782658963000 expected=1782658963000 bytes=53 (cap=16777216)
    PASS  non-ok-status:     msg="401 Unauthorized: {\"error\":\"invalid_token\"}"
    
    Total: 3/3 passed
    
  • Observed result after fix:
    • Negative control: server reported bytesSent=17825792 (17 MiB) but the runtime threw GitHub Copilot token refresh request: JSON response exceeds 16777216 bytes instead of buffering all 17 MiB. The error message embeds the cap value and the per-call label so logs can attribute the bounded-read to this call site (not chutes, not anthropic).
    • Happy path: a 53-byte token-refresh response parses cleanly through the bounded reader and yields access=copilot-token-XYZ with expires=1782658963000 (the 5-min bufferMs from resolveExpiresAtMsFromEpochSeconds is correctly applied).
    • Non-OK status: a 401 response with {"error":"invalid_token"} flows through readProviderTextResponse and is wrapped into the existing ${response.status} ${response.statusText}: ${text} error format — same observable behavior as before, with the body now bounded at 16 MiB.
  • What was not tested:
    • Did not exercise a live GitHub OAuth endpoint (a real server would never stream >16 MiB on these endpoints in the wild; the bounded-read contract is exactly about defending against adversarial/buggy servers, which the local Node server simulates faithfully).
    • Did not test the device-code polling path (lines 197 / 291 call sites) end-to-end — they share the same fetchJson helper so the wrapper-level guarantee is the same. Adding duplicate proof would inflate the PR without adding signal.
    • Did not test Node 24 specifically (Node 22 used locally; ReadableStream and TextEncoder behavior is consistent across Node 22 LTS and Node 24).

Out of scope

  • src/llm/utils/oauth/anthropic.ts:236 still has unbounded await response.text(). Different maintainer area; will be a follow-up PR with the same readProviderTextResponse swap.
  • google.ts SDK-level fetch injection was considered for B1 but is not feasible: @google/genai 2.7.0 exposes httpOptions: { baseUrl, apiVersion, headers, timeout, extraBody, retryOptions } but no fetch parameter at the GoogleGenAI constructor, and generateContentStream(params) does not accept a second RequestOptions argument, so the SDK cannot be wired through buildGuardedModelFetch.
  • E2 (mcp-http-fetch.ts:69) is a separate PR in the same campaign.

Diff stats

2 files changed, 91 insertions(+), 2 deletions(-)
  src/llm/utils/oauth/github-copilot.ts         +9/-2   (1 import + 2 line changes in fetchJson + 2 WHY-comment lines)
  src/llm/utils/oauth/github-copilot.test.ts    +82/-0  (3 new inline tests, no new test file, no committed proof script)

Size: S (≤ 100 LoC non-test). All test additions are inline in the existing github-copilot.test.ts.

Risk checklist

  • User-visible behavior change? Yes — same shape, same errors. The only new failure mode is GitHub Copilot <op>: JSON response exceeds 16777216 bytes when an OAuth endpoint streams >16 MiB. That is the desired safety behavior.
  • Config/env/migration change? No.
  • Security/auth/secrets/network change? Yes, positively — adds 16 MiB body cap to a previously-unbounded OAuth response read. No new attack surface; no new endpoints touched.
  • Highest-risk area: a misformed label could make the error unreadable. Mitigated by reusing the existing operation parameter (4 distinct, hand-formed strings already in production) and prefixing with GitHub Copilot so logs disambiguate from sibling bounded reads (chutes, anthropic, etc.).

Evidence

Real Node http server, real refreshGitHubCopilotToken wrapper, globalThis.fetch rewritten to point only the GitHub OAuth hostnames at the loopback server. Proof captured in one run, not committed (per checklist #23).

PASS  negative-control: serverGotRequest=true bytesSent=17825792 cap=16777216 msg="GitHub Copilot token refresh request: JSON response exceeds 16777216 bytes"
PASS  happy-path:        access=copilot-token-XYZ expires=1782658963000 expected=1782658963000 bytes=53 (cap=16777216)
PASS  non-ok-status:     msg="401 Unauthorized: {\"error\":\"invalid_token\"}"

Total: 3/3 passed

Byte-level quantification:

  • Server sent 17 MiB (17825792 bytes), client threw at the 16 MiB cap (16777216 bytes). bytesSent - cap = 1048576 (1 MiB overshoot, exactly one chunk) — proves the bounded reader stopped reading at the cap, not after buffering the full body.
  • Happy path: 53 bytes parses cleanly with expires = 1782658963000 = (expectedExpiry * 1000) - 5*60*1000 (5-min bufferMs from resolveExpiresAtMsFromEpochSeconds applied as expected).
  • 401 path: error format ${response.status} ${response.statusText}: ${text} preserved exactly.

Reproduction script (not committed):

node --import tsx /tmp/g1_real_server_proof.mts

Label: security

AI-assisted.

@openclaw-barnacle openclaw-barnacle Bot added size: S triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jun 28, 2026
@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 28, 2026, 11:47 AM ET / 15:47 UTC.

Summary
The PR routes GitHub Copilot OAuth/API fetchJson response bodies through shared 16 MiB bounded text/JSON readers and adds overflow, cancellation, and normal-response tests.

PR surface: Source +7, Tests +82. Total +89 across 2 files.

Reproducibility: yes. for source-level review. Current main and v2026.6.10 use unbounded response.text() and response.json() in the shared Copilot OAuth helper, and the PR body includes terminal proof from a real local Node server.

Review metrics: 2 noteworthy metrics.

  • OAuth reads capped: 4 operations through 1 helper. The cap decision covers device-code, device-token polling, token-refresh, and model-list responses from the shared Copilot helper.
  • Overlapping landing candidates: 1 open proof-positive PR. Maintainers should choose between this PR and fix(copilot-oauth): bound OAuth and API endpoint response reads #96877 before merge to avoid duplicate fixes.

Stored data model
Persistent data-model change detected: serialized state: src/llm/utils/oauth/github-copilot.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Risk before merge

  • [P1] The 16 MiB cap intentionally changes oversized Copilot device-code, token-refresh, and model-list responses from unbounded buffering to fail-closed errors, so maintainers should accept or tune that auth-provider boundary before merge.
  • [P1] fix(copilot-oauth): bound OAuth and API endpoint response reads #96877 is an open proof-positive implementation for the same helper; landing both would create duplicate implementations and review churn.

Maintainer options:

  1. Accept the shared provider readers (recommended)
    Land this PR as the canonical Copilot OAuth hardening if maintainers accept the 16 MiB fail-closed boundary, then close or retarget fix(copilot-oauth): bound OAuth and API endpoint response reads #96877.
  2. Tune the cap before merge
    If legitimate Copilot OAuth or model-list responses may exceed 16 MiB, adjust the cap or helper options and refresh overflow proof before landing either PR.
  3. Use the older direct-limit PR
    If maintainers prefer the direct readResponseWithLimit implementation in fix(copilot-oauth): bound OAuth and API endpoint response reads #96877, land that PR and close or retarget this PR.

Next step before merge

  • [P2] Manual review is appropriate because the remaining blocker is maintainer selection between two open Copilot OAuth cap PRs and acceptance or tuning of the cap, not a mechanical repair.

Security
Cleared: The diff is security-positive, replacing unbounded OAuth response buffering with existing bounded readers and adding no dependency, workflow, permission, package, or secret-handling changes.

Review details

Best possible solution:

Land one canonical bounded-read implementation for the core Copilot fetchJson helper, preferably the shared provider-reader shape if maintainers accept the 16 MiB boundary, then close or retarget the overlapping PR.

Do we have a high-confidence way to reproduce the issue?

Yes for source-level review. Current main and v2026.6.10 use unbounded response.text() and response.json() in the shared Copilot OAuth helper, and the PR body includes terminal proof from a real local Node server.

Is this the best way to solve the issue?

Yes for the implementation shape. Reusing the shared provider bounded readers at the single Copilot fetchJson helper is the narrowest maintainable fix I found; the remaining issue is maintainer coordination with the overlapping open PR and cap acceptance.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 2f851ecfe9df.

Label changes

Label justifications:

  • P2: This is normal-priority auth-provider availability and security hardening with limited Copilot-specific blast radius and no active outage evidence.
  • merge-risk: 🚨 compatibility: The patch intentionally changes oversized Copilot OAuth/API responses from unbounded buffering to a 16 MiB failure.
  • merge-risk: 🚨 auth-provider: The changed helper handles Copilot device login, token refresh, and model-list responses, so cap behavior can affect auth and provider setup flows.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes terminal output from a real local Node HTTP server exercising the production token-refresh wrapper against oversized, normal, and non-OK responses after the patch.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal output from a real local Node HTTP server exercising the production token-refresh wrapper against oversized, normal, and non-OK responses after the patch.
Evidence reviewed

PR surface:

Source +7, Tests +82. Total +89 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 9 2 +7
Tests 1 82 0 +82
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 91 2 +89

What I checked:

Likely related people:

  • vincentkoc: GitHub commit history for src/llm/utils/oauth/github-copilot.ts shows recent Copilot OAuth hardening, including bounded request timeouts and model-policy response cancellation. (role: recent area contributor; confidence: high; commits: 7b967c570109, 90ba9fc864a7; files: src/llm/utils/oauth/github-copilot.ts)
  • steipete: GitHub path history ties major Copilot OAuth/runtime extraction and safe expiry parsing work to this helper, including the refactor that moved provider OAuth helpers under src/llm. (role: feature-history contributor; confidence: high; commits: bb46b79d3c14, a5717c34ab0a, fb8b9e91380b; files: src/llm/utils/oauth/github-copilot.ts)
  • Alix-007: Authored merged provider JSON bounded-read work and adjacent GitHub Copilot usage response hardening using the same shared helper pattern. (role: adjacent bounded-read contributor; confidence: medium; commits: 2592f8a51a4e, 646e54ae3578; files: src/agents/provider-http-errors.ts, extensions/github-copilot/usage.ts, extensions/github-copilot/models.test.ts)
  • joshavant: Recent history for src/agents/provider-http-errors.ts includes successful provider response read bounding, which is the helper family this PR reuses. (role: shared helper contributor; confidence: medium; commits: 0a14444924e3; files: src/agents/provider-http-errors.ts)
  • hugenshen: Authored merged GitHub Copilot model discovery and embeddings JSON bounded-read hardening on another Copilot provider surface. (role: adjacent Copilot response-hardening contributor; confidence: medium; commits: 1aa7cafc35a1, 499c30c254e7; files: extensions/github-copilot/embeddings.ts, extensions/github-copilot/embeddings.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 auth-provider 🚨 May break OAuth, tokens, provider routing, model choice, or credentials. labels Jun 28, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 28, 2026
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper review

@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

Thanks @vincentkoc for the merge and the P1 prioritization on the OAuth surface.

Two things from this round I'd like to call out as useful signals for me:

  1. The readProviderTextResponse / readProviderJsonResponse per-call label pattern (e.g. "GitHub Copilot token refresh request") landed cleanly because you and the rest of the OAuth maintainers had already established the 16 MiB cap in src/agents/provider-http-errors.ts — I was just wiring a new surface into existing seams, not inventing a new pattern. That made the diff small and the review deterministic.

  2. The re-review loop on the PR body 4-section requirement (post-fix(oauth): bound github-copilot OAuth response reads at 16 MiB #97499 itself — see the new ## Evidence heading) is now baked into my pre-submit checklist. Won't need to trigger that on future OAuth bounded-reads.

Sibling surface src/llm/utils/oauth/anthropic.ts:236 is next, scoped to the same helper. I'll keep the diff size and shape the same as this one.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 29, 2026
…claw#97499)

* fix(oauth): bound github-copilot OAuth response reads at 16 MiB

* chore: trigger CI re-run after PR body update
QiuYuang pushed a commit to QiuYuang/openclaw that referenced this pull request Jul 1, 2026
…claw#97499)

* fix(oauth): bound github-copilot OAuth response reads at 16 MiB

* chore: trigger CI re-run after PR body update
chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 1, 2026
…claw#97499)

* fix(oauth): bound github-copilot OAuth response reads at 16 MiB

* chore: trigger CI re-run after PR body update
Rorqualx pushed a commit to Rorqualx/cortex that referenced this pull request Jul 2, 2026
…claw#97499)

* fix(oauth): bound github-copilot OAuth response reads at 16 MiB

* chore: trigger CI re-run after PR body update

(cherry picked from commit bd0c052)
chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 3, 2026
…claw#97499)

* fix(oauth): bound github-copilot OAuth response reads at 16 MiB

* chore: trigger CI re-run after PR body update
wheakerd pushed a commit to wheakerd/clawdbot that referenced this pull request Jul 15, 2026
…claw#97499)

* fix(oauth): bound github-copilot OAuth response reads at 16 MiB

* chore: trigger CI re-run after PR body update

(cherry picked from commit bd0c052)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 auth-provider 🚨 May break OAuth, tokens, provider routing, model choice, or credentials. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants