Skip to content

Commit fafad20

Browse files
simurg79Bertan Ariedelauna
authored
fix(vscode-lm): integrate guarded recovery into streaming responses (#1608)
* fix(vscode-lm): sanitize surrogates, recover leaked tool calls, and window-safe tool_result truncation Hardens the VS Code Language Model provider (notably GitHub Copilot serving Anthropic Claude) against three failure modes: - Surrogate sanitization: a lone UTF-16 surrogate cannot be encoded as UTF-8, so the backend rejects the entire request with a 400. sanitizeSurrogates() replaces unpaired surrogates with U+FFFD while preserving valid pairs (emoji, CJK ext.), applied to string messages, tool results, and text parts. - Leaked tool-call recovery: some backends stream a tool call as raw <invoke> XML instead of a structured LanguageModelToolCallPart, leaving the turn with no tool_use block and stalling the task in a "no tools used" retry loop. extractLeakedToolCalls() and trailingPartialToolMarkerLength() detect the markup mid-stream (including markers split across chunk boundaries) and replay it as a real tool call, conservatively: only for <invoke> names matching a tool actually offered that turn, and only when tools were offered. - Window-safe tool_result truncation: Copilot's backend trims over-window requests without preserving tool_use/tool_result pairing, orphaning a tool_result and causing a 400 (unexpected tool_use_id). truncateToolResultsToFitWindow() and middleOutTruncate() shrink oversized tool_result payloads on our side (largest first, middle-out, pairing preserved) before sending. Ported from simurg79/Roo-Code#12. * test(vscode-lm): cover leaked tool-call salvage and tool_result truncation paths Raises patch coverage on the new vscode-lm reliability code above the 80%% codecov/patch gate by exercising the streaming salvage state machine (marker split across chunks, multi-chunk buffering, unknown-tool passthrough, carried tail) and the tool_result truncation helpers (array-form content, surrogate-safe middle-out, guard clauses). * fix(vscode-lm): guard leaked tool-call recovery against quoted markup Address review feedback on the leaked-tool-call salvage path: a tool name alone was not a sufficient gate, so prose or fenced examples reproducing the invoke markup could be replayed as real calls. Adds the quoted/fenced guard plus coverage. Also records the empirical vscode.lm probe as a project skill (probe-vscode-lm-api) with the scratch probe extension, the false-positive replay harness, representative transcripts, and the consent-gate gotcha. * chore(knip): exclude .roo skill assets from unused-file analysis Skill directories hold reference scripts and captured artifacts that are intentionally never imported by the build. * fix(vscode-lm): address review feedback on leaked tool-call recovery - dispose the probe CancellationTokenSource in a finally block * docs(skill): drop probe transcripts from repo Remove the ~120KB raw probe transcript corpus from the vscode-lm probe skill; keep the measured findings and their stated limits in SKILL.md. * chore: move probe skill scripts under scripts/, drop .roo knip ignore * fix(vscode-lm): harden quoted-markup detection and bound the salvage buffer Loop tag stripping until stable so `<<script>>` cannot reconstruct a tag after a single pass (CodeQL incomplete multi-character sanitization). Track fence marker and width instead of counting ``` runs for parity, so tilde fences and 4+ backtick fences are recognized. Treat a quoted invoke that ends its line as quoted when an explicit quoting cue precedes it, rather than recovering it as a live tool call. Keying off leading prose alone was tried previously and regressed genuine recoveries, so the cue is deliberately narrow. Bound the salvage buffer so markup that never closes is flushed as plain text instead of withholding the response until the stream ends. * test(vscode-lm): assert the salvage buffer flushes mid-stream The first version of this test only checked the flushed text's content, which the end-of-stream drain produces even without the cap, so it passed against the unfixed code. Assert instead that text reaches the consumer before the stream is exhausted, which is what the bound actually changes. * test(vscode-lm): cover fence-close branch of isInsideCodeFence Replace the vacuous four-backtick test with a nested inner-fence case and add a closed-fence recovery test, both of which fail under the old backtick-parity counting. * fix(vscode-lm): clamp messages budget to a positive floor Address CodeRabbit review: a system prompt or tool schema large enough to consume the derived char budget left messagesBudgetChars non-positive, which made truncateToolResultsToFitWindow a no-op exactly when the request was most oversized. Clamp to MIN_TOOL_RESULT_CHARS and cover it with a regression test. Also reattach a misplaced doc comment and dedupe a test helper. * fix(vscode-lm): require function_calls wrapper and sanitize tool-call args Addresses taltas review feedback on PR #1188. * chore: remove probe skill and harness from PR * docs: drop dangling probe skill path from vscode-lm comment * fix(vscode-lm): harden streaming tool-call recovery and token budgeting Recover only wrapped function_calls/invoke markup leaked into text parts; bare unwrapped invoke is passed through unchanged. Add narrow top-level schema-aware parameter conversion and an approximate output-budget guard, with expanded provider unit tests. * fix: derive mutation gate base from the PR merge commit's first parent GitHub checks out the synthetic pull request merge commit as github.sha, but pull_request.base.sha is frozen when the event is created. Once main advances, the stale base made the changed-code mutation gate attribute unrelated upstream-only files to the pull request (3294 changed executable lines across 87 files instead of 361 across the 2 files the PR actually touches). Resolve the base from the checked-out head's first parent when the head is a merge commit, leaving non-merge heads and the merge_group path unchanged. Head stays github.sha so selector coordinates remain aligned with the checked-out tree. * fix(vscode-lm): admit requests against the raw context budget, not the trimming floor The clamp to MIN_TOOL_RESULT_CHARS exists only to keep tool_result trimming productive; using it for the final admission check let a request through whenever the raw budget was positive but below the floor, sending an over-window request. Judge admission against the raw budget and cover the boundary with a regression test. Also guarantee temp-repository cleanup in the two stryker-diff pull-request-selection tests via try/finally, and move the system-prompt surrogate sanitization test out of the leaked streaming recovery group. * fix(vscode-lm): accept an explicit null for a nullable leaked-tool parameter declaredParamType stripped "null" from a declared ["T","null"] union, so convertLeakedParamValue rejected a literal JSON null and failed the whole leaked block closed to text. It now reports that null is permitted and the conversion consults that flag. A non-nullable object still rejects null, and a declared string keeps the literal text "null". Also assert the streamed text chunk in the accepted-budget test, which previously drained the stream and only checked the sendRequest call. * fix(vscode-lm): support null-only parameter schemas in leaked tool-call recovery Handle both structured type: "null" and array type: ["null"] forms in declaredParamType so recovery emits JSON null, while continuing to fail closed for non-null values. Adds unit coverage for both helper forms and a createMessage runtime regression test with a mocked VS Code LM host. * refactor(vscode-lm): narrow this branch to leaked tool-call recovery Surrogate sanitization and context-window tool_result truncation are being proposed as independent changes, so remove them here. Recovery does not depend on either: it keeps the original unsanitized system-prompt boundary and no longer references the truncation helpers. Retains the null-only parameter schema fix and the stryker-diff CI prerequisite. * refactor(vscode-lm): drop surrogate sanitization from the transform layer Sanitization is proposed independently, so restore src/api/transform to origin/main here. Recovery does not use it; the full provider and transform suites pass without it. * refactor(vscode-lm): defer leaked tool-call streaming integration Keeps the complete leaked tool-call parser and its direct tests, but removes the createMessage streaming integration and its integration tests so the changed-code mutation gate stays within its per-run mutant budget. createMessage is restored byte-for-byte to the base implementation, so the parser is present but not yet activated; a follow-up change re-enables it. * feat(vscode-lm): recover leaked tool calls during streaming Re-enables the deferred leaked tool-call parser inside createMessage: streaming salvage state, start-marker detection with partial-marker carry across chunks, buffering until the invoke block completes, an overflow fallback that releases unclosed markup as text, and an ordered flush that emits prose before any recovered call and runs before native tool calls. Restores the streaming integration tests. Depends on the parent parser change; together they reproduce the original behavior exactly. * fix(vscode-lm): remove unreachable empty-buffer guard in flushSalvage * fix: preserve large leaked calls and exercise fence guard (#1608) * fix(vscode-lm): drain decided salvage prefixes past the buffer cap A complete invoke block in the buffer bypassed the cap entirely, so once the buffer grew past it both the recovered call and the trailing prose were withheld until the stream ended. Drain only the decided prefix and retain the in-flight tail, keeping salvageEmittedText aligned so quoting context survives. * test(vscode-lm): cover empty-emit side of the carried marker guard * fix(vscode-lm): keep narration cue in the decided span past the salvage cap Cutting the drained span at the last </invoke> dropped a trailing quoting cue, so an over-cap stream recovered a call the one-shot path refuses. Extend the span to the end of that line and wait while the line is still open. Also assert distinct ids for multiple recovered calls. * fix(vscode-lm): keep an open invoke buffered when the salvage drain boundary falls inside it * fix(vscode-lm): find a pending invoke opener that spans the drain newline * fix(vscode-lm): defer leaked-call salvage to segment boundaries for chunk-invariant output * test(vscode-lm): cover narration newline variant and pin failure-path behaviour * test(vscode-lm): exercise opener whitespace beyond the old carry limit in split fuzz * test(vscode-lm): wrap unknown-tool block so the offered-name guard decides the result * perf(vscode-lm): skip rescans while carrying invoke whitespace; cover no-tools and fenced-prefix paths * fix(vscode-lm): recover leaked calls that use the declared encoded tool name --------- Co-authored-by: Bertan Ari <[email protected]> Co-authored-by: edelauna <[email protected]>
1 parent 89a90f5 commit fafad20

2 files changed

Lines changed: 1531 additions & 10 deletions

File tree

0 commit comments

Comments
 (0)