Skip to content

fix(github-copilot): sanitize unsafe reasoning replay ids#83221

Merged
galiniliev merged 2 commits into
openclaw:mainfrom
galiniliev:bug-019-copilot-replay-reasoning-ids
May 18, 2026
Merged

fix(github-copilot): sanitize unsafe reasoning replay ids#83221
galiniliev merged 2 commits into
openclaw:mainfrom
galiniliev:bug-019-copilot-replay-reasoning-ids

Conversation

@galiniliev

@galiniliev galiniliev commented May 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: GitHub Copilot native Responses replay could still send provider-generated reasoning replay items with overlong/non-replayable IDs after tool-call replay IDs were repaired.
  • Why it matters: Copilot rejects this request body before generation with opaque invalid_request_body schema failures, so affected web chat and scheduled agent turns cannot continue.
  • What changed: the GitHub Copilot adapter now sanitizes Copilot replay IDs at the provider boundary: it preserves valid reasoning replay IDs, drops unsafe reasoning replay items instead of stripping their IDs, and continues rewriting message/function-call connection-bound IDs.
  • What did NOT change (scope boundary): shared OpenAI Responses replay behavior is unchanged; this does not change non-Copilot providers, provider routing, session-file repair, or tool-call pairing semantics.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Real behavior proof (required for external PRs)

  • Behavior addressed: GitHub Copilot native Responses replay no longer sends invalid overlong reasoning.id values after overlong replay tool IDs are normalized; unsafe Copilot reasoning replay items are removed at the Copilot adapter boundary.
  • Real environment tested: Local Windows source worktree using Node v24.15.0, OpenClaw source branch bug-019-copilot-replay-reasoning-ids, model route represented as github-copilot/gpt-5.5 native Responses.
  • Exact steps or command run after this patch: node node_modules\vitest\vitest.mjs run extensions\github-copilot\connection-bound-ids.test.ts -t "github-copilot connection-bound response IDs" --reporter=verbose --pool=threads --maxWorkers=1 --testTimeout=30000; then node node_modules\vitest\vitest.mjs run extensions\github-copilot\stream.test.ts -t "sanitizes reasoning replay" --reporter=verbose --pool=threads --maxWorkers=1 --testTimeout=30000; then node node_modules\vitest\vitest.mjs run src\agents\openai-transport-stream.test.ts -t "overlong Copilot Responses replay" --reporter=verbose --pool=threads --maxWorkers=1 --testTimeout=30000; then git diff --check.
  • Evidence after fix: Terminal capture, with local checkout path redacted:
RUN  v4.1.6 [redacted local worktree]
✓ |extension-providers| ../../extensions/github-copilot/connection-bound-ids.test.ts > github-copilot connection-bound response IDs > rewrites opaque message response item IDs deterministically 105ms
✓ |extension-providers| ../../extensions/github-copilot/connection-bound-ids.test.ts > github-copilot connection-bound response IDs > uses response item type prefixes and preserves local IDs 1ms
✓ |extension-providers| ../../extensions/github-copilot/connection-bound-ids.test.ts > github-copilot connection-bound response IDs > preserves valid reasoning IDs regardless of encrypted_content 1ms
✓ |extension-providers| ../../extensions/github-copilot/connection-bound-ids.test.ts > github-copilot connection-bound response IDs > preserves valid base64-ish reasoning IDs with and without encrypted content 1ms
✓ |extension-providers| ../../extensions/github-copilot/connection-bound-ids.test.ts > github-copilot connection-bound response IDs > drops unsafe reasoning replay items instead of stripping their IDs 0ms
✓ |extension-providers| ../../extensions/github-copilot/connection-bound-ids.test.ts > github-copilot connection-bound response IDs > patches response payload input arrays only 1ms
Test Files  1 passed (1)
Tests  6 passed (6)

RUN  v4.1.6 [redacted local worktree]
✓ |extension-providers| ../../extensions/github-copilot/stream.test.ts > wrapCopilotAnthropicStream > adds Copilot headers, sanitizes reasoning replay, and rewrites message IDs before payload send 23ms
Test Files  1 passed (1)
Tests  1 passed | 6 skipped (7)

RUN  v4.1.6 [redacted local worktree]
✓ |agents-core| ../../src/agents/openai-transport-stream.test.ts > openai transport stream > normalizes overlong Copilot Responses replay tool ids before dispatch 151ms
✓ |agents-core| ../../src/agents/openai-transport-stream.test.ts > openai transport stream > keeps distinct overlong Copilot Responses replay item ids distinct 3ms
Test Files  2 passed (2)
Tests  4 passed | 328 skipped (332)

git diff --check
(no output; exit 0)
  • Observed result after fix: The Copilot sanitizer preserves valid provider-issued reasoning IDs up to 64 characters, including base64-ish characters such as +, /, and =, drops overlong/missing/non-string reasoning replay items as whole items, and still rewrites message/function-call connection-bound IDs to stable Copilot-safe IDs.
  • What was not tested: A live Copilot replay against the private affected agent session was not rerun because it requires private session history and provider credentials. Broad pnpm check:changed was not run locally because this is a Codex worktree. oxfmt --write was attempted on the touched files but failed in this worktree with Windows spawn EPERM; git diff --check passed.
  • Before evidence (optional but encouraged): Redacted runtime evidence from 2026-05-17 and 2026-05-18 showed at least 20 post-fix GitHub Copilot native Responses failures from 2026-05-17 06:40 UTC through 2026-05-18 00:45 UTC. Copilot returned 400 {"message":"","code":"invalid_request_body"} with providerRuntimeFailureKind="schema" and stable failure hash sha256:3c08634b4d9f. A sanitized structural probe of the affected replay showed functionCalls=114, functionOutputs=114, callIdsOver64=0, orphanOutputs=0, and duplicateIds=0, but still had idsOver64=53, all on reasoning items, with maxIdLen=408. The first offending item was same-provider Copilot native Responses reasoning with keys encrypted_content,id,summary,type and non-safe/overlong ID content.

Root Cause (if applicable)

  • Root cause: shared Responses replay rebuilt same-provider reasoning items from persisted thinkingSignature JSON, while the Copilot adapter's existing connection-bound ID sanitizer intentionally skipped reasoning items. After the prior tool-ID repair, overlong function-call IDs were fixed but overlong Copilot reasoning item IDs still reached the native Responses request body.
  • Missing detection / guardrail: Existing Copilot replay tests covered overlong function-call IDs and connection-bound message IDs, but not Copilot reasoning replay items with encrypted content and invalid IDs.
  • Contributing context (if known): Copilot reports this invalid request shape as an opaque invalid_request_body, so once tool IDs were repaired the remaining reasoning ID failures no longer named the exact offending field.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: extensions/github-copilot/connection-bound-ids.test.ts; extensions/github-copilot/stream.test.ts; adjacent unchanged coverage in src/agents/openai-transport-stream.test.ts.
  • Scenario the test should lock in: Copilot Responses replay preserves valid provider-issued reasoning IDs, drops unsafe reasoning replay items as whole items, and still rewrites message/function-call connection-bound IDs before payload send.
  • Why this is the smallest reliable guardrail: The failure occurs at the Copilot provider payload boundary before any model generation, and the adapter hook is the narrowest place that owns Copilot-specific replay ID rules.
  • Existing test that already covers this (if any): Existing overlong Copilot replay tool-ID tests still cover the shared tool-call normalization half.
  • If no new test is added, why not: N/A

User-visible / Behavior Changes

Copilot-backed agent sessions with invalid persisted reasoning replay item IDs should resume instead of failing before generation with a schema/tool-payload rejection. Valid Copilot reasoning replay IDs are still preserved.

Diagram (if applicable)

Before:
[persisted Copilot reasoning item id >64 chars] -> [shared Responses payload] -> [Copilot invalid_request_body]

After:
[persisted Copilot reasoning item id >64 chars] -> [Copilot adapter drops unsafe reasoning replay item] -> [provider-valid replay payload]

Security Impact (required)

  • New permissions/capabilities? (Yes/No) No
  • Secrets/tokens handling changed? (Yes/No) No
  • New/changed network calls? (Yes/No) No
  • Command/tool execution surface changed? (Yes/No) No
  • Data access scope changed? (Yes/No) No
  • If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

  • OS: Windows source worktree for tests; original failing runtime setup identifiers redacted.
  • Runtime/container: Node v24.15.0 local source checkout.
  • Model/provider: github-copilot/gpt-5.5 native Responses.
  • Integration/channel (if any): Agent runtime replay; no external channel surface changed.
  • Relevant config (redacted): Provider route github-copilot + openai-responses; private agent/session identifiers omitted.

Steps

  1. Build or intercept a Copilot Responses request from replay history containing an overlong provider-generated reasoning item ID and overlong tool-call IDs.
  2. Run the Copilot adapter payload sanitizer before dispatch.
  3. Verify the generated input payload and run the focused Vitest filters listed above.

Expected

  • Unsafe Copilot reasoning replay items with missing, non-string, or overlong IDs are dropped as whole items.
  • Valid Copilot reasoning.id values of length 1-64 are preserved, including base64-ish characters.
  • Non-reasoning connection-bound IDs are rewritten to stable provider-safe IDs, and function-call outputs remain paired with normalized function-call IDs.

Actual

  • Before this patch, the payload still contained 53 overlong reasoning.id values in the affected session structure after tool-call IDs had been normalized.
  • After this patch, the Copilot adapter removes unsafe reasoning replay items while preserving valid reasoning replay and existing tool-call/message ID normalization.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: Copilot replay ID sanitizer unit coverage; Copilot Responses wrapper payload-hook coverage; existing overlong Copilot replay tool-ID tests; git diff --check.
  • Edge cases checked: Valid short reasoning IDs with and without encrypted_content; base64-ish valid IDs containing +, /, and =; overlong, missing, and non-string reasoning IDs; deterministic rewriting for message/function-call IDs.
  • What you did not verify: Live Copilot replay against the private affected session, broad changed gates, and formatter completion due to the local spawn EPERM blocker noted above.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.

Compatibility / Migration

  • Backward compatible? (Yes/No) Yes
  • Config/env changes? (Yes/No) No
  • Migration needed? (Yes/No) No
  • If yes, exact upgrade steps: N/A

Risks and Mitigations

  • Risk: Dropping unsafe reasoning replay items can reduce reasoning-state continuity for a repaired Copilot turn.
    • Mitigation: the adapter only drops reasoning items whose ID cannot be replayed safely for Copilot; valid provider-issued reasoning IDs are preserved, and invalid ones already caused request rejection before generation.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S maintainer Maintainer-authored PR labels May 17, 2026
@clawsweeper

clawsweeper Bot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge.

Workflow note: Future ClawSweeper reviews update this same comment in place.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by maintainer comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors can comment @clawsweeper re-review or @clawsweeper re-run on their own open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Summary
The PR updates the GitHub Copilot provider payload sanitizer to drop invalid reasoning replay items, preserves valid reasoning IDs and existing message/function-call ID rewrites, adds focused tests, and records a changelog fix.

Reproducibility: yes. at source level: current main replays persisted reasoning items with IDs intact and the Copilot sanitizer skips reasoning items, while the linked report provides redacted runtime failures and a structural probe showing overlong reasoning IDs. I did not run a live Copilot replay in this read-only review.

PR rating
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🐚 platinum hermit
Summary: The implementation looks focused and likely correct, but the missing real behavior proof keeps the PR out of merge-ready shape.

Rank-up moves:

  • Add redacted after-fix Copilot replay proof showing the outbound request is accepted, or get an explicit maintainer proof override.
  • Run the focused repo wrapper or Testbox checks maintainers require before merge.
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Real behavior proof
Needs real behavior proof before merge: The PR body shows after-fix Vitest terminal output, but external PR proof still needs redacted live output/logs or a maintainer override; contributors should redact private details and update the PR body to trigger re-review.

Next step before merge
Protected maintainer label plus missing after-fix real Copilot proof require human merge/proof judgment; there is no narrow automated repair finding to queue.

Security
Cleared: The diff only changes Copilot request-payload sanitization, focused tests, and changelog text; no concrete security or supply-chain concern was found.

Review details

Best possible solution:

Land the provider-boundary sanitizer after maintainer review and either redacted after-fix Copilot replay proof or an explicit proof override, then let the linked bug close on merge.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main replays persisted reasoning items with IDs intact and the Copilot sanitizer skips reasoning items, while the linked report provides redacted runtime failures and a structural probe showing overlong reasoning IDs. I did not run a live Copilot replay in this read-only review.

Is this the best way to solve the issue?

Yes. Sanitizing only GitHub Copilot replay IDs at the provider payload boundary is the narrowest maintainable fix and avoids changing shared OpenAI Responses replay behavior or session-file migration.

What I checked:

  • Current main replays reasoning IDs unchanged: Current main parses JSON thinkingSignature reasoning items and pushes the resulting item into Responses input with its id intact when replay item IDs are enabled. (src/agents/openai-transport-stream.ts:860, bc4f27c89a3c)
  • Current Copilot sanitizer skips reasoning items: Current main rewrites connection-bound message/function-call IDs but explicitly continues for type === "reasoning", leaving overlong reasoning IDs in the Copilot request body. (extensions/github-copilot/connection-bound-ids.ts:42, bc4f27c89a3c)
  • PR implements provider-boundary sanitization: The PR commit adds sanitizeCopilotReplayResponseIds, preserving non-empty reasoning IDs up to 64 characters and splicing out unsafe reasoning items before message/function-call ID rewriting continues. (extensions/github-copilot/connection-bound-ids.ts:36, 9756e06a99fe)
  • PR adds focused regression coverage: The PR adds unit coverage for valid base64-ish reasoning IDs, overlong/missing/non-string reasoning IDs, and the Copilot Responses stream payload hook dropping unsafe reasoning items before dispatch. (extensions/github-copilot/connection-bound-ids.test.ts:55, 9756e06a99fe)
  • Proof remains mock-only: The PR body includes after-fix focused Vitest terminal output and redacted before-failure evidence, but it does not include an after-fix live Copilot replay, outbound request log, or provider response showing the real path succeeds.
  • Protected label and linked issue: The provided GitHub context shows the PR has the protected maintainer label and closes the still-open linked bug report at [Bug]: Copilot replay repair leaves invalid reasoning item IDs #83220, so cleanup should keep it open for maintainer handling.

Likely related people:

  • Gio Della-Libera: git blame shows the current Copilot connection-bound ID helper and stream wrapper lines in commit fb53c2d. (role: introduced current helper in local history; confidence: medium; commits: fb53c2d61092; files: extensions/github-copilot/connection-bound-ids.ts, extensions/github-copilot/stream.ts, src/agents/openai-transport-stream.ts)
  • @vincentkoc: Path history shows recent work on Copilot runtime headers and shared provider payload patch helpers that this sanitizer uses. (role: adjacent provider stream contributor; confidence: medium; commits: cdccbf2c1c8e, cc1881a83846; files: extensions/github-copilot/stream.ts, extensions/github-copilot/connection-bound-ids.ts)
  • @steipete: Shortlog and path history show the largest recent share of edits across the Copilot provider stream/tests and OpenAI transport area, including provider stream SDK seam work. (role: recent area contributor; confidence: medium; commits: be526d642312, 5d20c73e05c8; files: extensions/github-copilot/stream.ts, src/agents/openai-transport-stream.ts)

Remaining risk / open question:

  • No after-fix live Copilot replay/provider-response proof is attached; a maintainer may need to override the gate if the private credentials or session history cannot be shared safely.
  • Broad changed checks are not shown in the PR body; the listed validation is focused Vitest filters plus git diff --check.

Codex review notes: model gpt-5.5, reasoning high; reviewed against bc4f27c89a3c.

@clawsweeper clawsweeper Bot added P1 High-priority user-facing bug, regression, or broken workflow. impact:session-state Session, memory, transcript, context, or agent state can drift or corrupt. impact:auth-provider Auth, provider routing, model choice, or SecretRef resolution may break. labels May 17, 2026
@galiniliev
galiniliev force-pushed the bug-019-copilot-replay-reasoning-ids branch from 544e3dd to 6c89034 Compare May 18, 2026 01:59
@openclaw-barnacle openclaw-barnacle Bot added extensions: github-copilot and removed agents Agent runtime and tooling labels May 18, 2026
@galiniliev galiniliev changed the title fix(agents): strip invalid Copilot reasoning replay ids fix(github-copilot): sanitize unsafe reasoning replay ids May 18, 2026
@clawsweeper clawsweeper Bot added the rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. label May 18, 2026
@galiniliev
galiniliev force-pushed the bug-019-copilot-replay-reasoning-ids branch from 6c89034 to b481eec Compare May 18, 2026 02:40
@galiniliev

Copy link
Copy Markdown
Contributor Author

Maintainer verification for source head b481eec against base upstream/main bc4f27c89a:

git diff --check upstream/main...HEAD
node scripts/check-changelog-attributions.mjs CHANGELOG.md
node scripts/run-vitest.mjs extensions/github-copilot/connection-bound-ids.test.ts extensions/github-copilot/stream.test.ts src/agents/openai-transport-stream.test.ts -- -t "github-copilot connection-bound response IDs|sanitizes reasoning replay|overlong Copilot Responses replay"

Result: all commands passed. The focused Vitest command covered 4 files / 349 tests, including the Copilot connection-bound ID sanitizer, the Copilot Responses stream hook, and the core replay regression case.

The Real behavior proof parser also passes for the PR body after the proof-field formatting fix, and the current PR head has no failed or pending GitHub checks.

Known proof gap: I did not run a live GitHub Copilot replay against the original private affected session because that requires private session state and provider credentials. The landing proof is the redacted before evidence plus provider-boundary sanitizer coverage that reproduces and blocks the overlong reasoning replay item shape.

@galiniliev
galiniliev merged commit ad55d48 into openclaw:main May 18, 2026
107 checks passed
@galiniliev

Copy link
Copy Markdown
Contributor Author

Landed via squash merge as ad55d48 from source head b481eec.

This lands the GitHub Copilot native Responses replay sanitizer for unsafe reasoning item IDs, plus focused regression coverage and the changelog entry.

galiniliev added a commit to galiniliev/openclaw that referenced this pull request May 20, 2026
SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 24, 2026
SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 24, 2026
SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 24, 2026
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
galiniliev added a commit to galiniliev/openclaw that referenced this pull request May 25, 2026
SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 26, 2026
SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 26, 2026
SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 26, 2026
jameslcowan pushed a commit to jameslcowan/openclaw that referenced this pull request Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: github-copilot impact:auth-provider Auth, provider routing, model choice, or SecretRef resolution may break. impact:session-state Session, memory, transcript, context, or agent state can drift or corrupt. maintainer Maintainer-authored PR P1 High-priority user-facing bug, regression, or broken workflow. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Copilot replay repair leaves invalid reasoning item IDs

2 participants