Skip to content

Commit ed3e8ec

Browse files
author
Bertan Ari
committed
fix(vscode-lm): guard leaked tool-call recovery against quoted markup
Address review feedback on the leaked-tool-call salvage path: a tool name alone was not a sufficient gate, so prose or fenced examples reproducing the invoke markup could be replayed as real calls. Adds the quoted/fenced guard plus coverage. Also records the empirical vscode.lm probe as a project skill (probe-vscode-lm-api) with the scratch probe extension, the false-positive replay harness, representative transcripts, and the consent-gate gotcha.
1 parent 306976d commit ed3e8ec

23 files changed

Lines changed: 4593 additions & 40 deletions
Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
---
2+
name: probe-vscode-lm-api
3+
description: How to empirically probe the VS Code Language Model API (`vscode.lm`) with a scratch extension against a real extension host, and the measured findings about Copilot Claude models leaking tool-call markup into text. Use when asked to "test the vscode.lm API", "probe Copilot model behavior", "capture a raw LM transcript", "does the model leak tool-call markup", "verify LanguageModelToolCallPart behavior", or when reasoning about `extractLeakedToolCalls()` in the vscode-lm provider.
4+
---
5+
6+
# Probing the VS Code LM API Empirically
7+
8+
## When to Use This Skill
9+
10+
- A claim is being made about what a Copilot-backed model _actually_ emits over `vscode.lm` (tool-call parts vs. text), and it needs evidence rather than inference.
11+
- Changing [`extractLeakedToolCalls()`](src/api/providers/vscode-lm.ts) or its guards, and you need a false-positive corpus.
12+
- Any question that can only be answered by real `model.sendRequest()` traffic — the mocked unit tests cannot answer it.
13+
14+
## When NOT to Use This Skill
15+
16+
- Ordinary provider work covered by [`src/api/providers/__tests__/vscode-lm.spec.ts`](src/api/providers/__tests__/vscode-lm.spec.ts). Live probing is slow and burns Copilot quota.
17+
- Anything about non-`vscode-lm` providers. Anthropic-API behavior does not transfer.
18+
19+
## Running the Probe
20+
21+
Scripts live in [`scripts/`](.roo/skills/probe-vscode-lm-api/scripts/) next to this file.
22+
23+
1. Copy `scripts/package.json` and `scripts/extension.js` into a scratch directory, e.g. `<repo>\.tmp\lmprobe\`. No build, no `npm install` — it is plain CommonJS against the `vscode` module.
24+
2. Adjust `OUT_DIR` at the top of `extension.js` to the transcript output directory.
25+
3. Launch a **new** extension host window:
26+
27+
```
28+
code --extensionDevelopmentPath=<repo>\.tmp\lmprobe --new-window <repo>
29+
```
30+
31+
4. In that new window: `Ctrl+Shift+P` -> **LM Probe: Run** (or click "Run probe" on the toast).
32+
5. Wait for the completion notification. Transcripts and `summary.json` land in `OUT_DIR`.
33+
34+
Each run writes a `.json` of every stream part and a `.txt` of the exact concatenated text, named `<modelId>__<scenario>__run<N>`.
35+
36+
### Delegate the UI driving
37+
38+
Steps 3-5 involve a live window. Delegate them to **`ui-operator`** mode rather than doing them inline; screenshots and control trees consume large amounts of context.
39+
40+
## Gotchas
41+
42+
### The consent gate needs a real user gesture
43+
44+
**Do not call `model.sendRequest()` from `activate()`.** Every request fails with:
45+
46+
```
47+
Language model '<model-id>' cannot be used by '<publisher>.<ext-id>'
48+
```
49+
50+
This is not a quota, auth, or manifest problem — `vscode.lm` grants consent only in response to a genuine user gesture. The probe must therefore be triggered from the Command Palette (or a notification button click). This is the single most expensive trap here; it silently fails 100% of requests and looks like an entitlement bug.
51+
52+
### Never `Stop-Process` filtered on window title
53+
54+
**WARNING:** During this experiment, killing processes matched by window title destroyed the user's unrelated VS Code windows and their unsaved work.
55+
56+
Safe alternative: only ever _launch_ new windows with `--new-window`, and close the probe window by hand. Never bulk-terminate `Code.exe` by title, `MainWindowTitle`, or any other fuzzy match.
57+
58+
### Run tests with pnpm, not npx
59+
60+
```
61+
pnpm --dir src exec vitest run <path>
62+
```
63+
64+
Never `npx vitest` — it resolves a wrong hoisted 3.2.4 instead of the pinned 4.1.9 and produces phantom failures.
65+
66+
## Measured Findings
67+
68+
Sample: 210 live requests, 7 Copilot Claude models x 6 scenarios x 5 repeats, 0 errors. Raw evidence in [`transcripts/`](.roo/skills/probe-vscode-lm-api/transcripts/).
69+
70+
| Scenario | Setup | Runs | `<invoke` in text |
71+
| -------- | ----------------------------------------- | ---- | ----------------- |
72+
| A | tools declared + agent system prompt | 35 | 0 |
73+
| B | tools declared, no system prompt | 35 | 0 |
74+
| C | tools declared + ~300KB filler context | 35 | 0 |
75+
| D | no tools, model asked to emit the markup | 35 | 14 |
76+
| E | asked to quote the markup in prose | 35 | 23 |
77+
| F | asked to quote the markup in a code fence | 35 | 21 |
78+
79+
Observations:
80+
81+
- **The leak did not reproduce.** 105/105 tool-declared runs (A+B+C) emitted a proper `LanguageModelToolCallPart` and leaked nothing into text parts. This bounds the leak rate at a low value; it is **not** proof of absence. 105 runs across 7 models cannot exclude a rare or prompt-specific trigger.
82+
- **Wrapped vs. bare inverts the intuition.** All 14 genuine emitted invocations (D) were wrapped in `<function_calls>`; 0 were bare. All 44 quoted-in-prose cases (E+F) were bare; 0 were wrapped. In this sample, _bare correlates with quoting and wrapped with genuine invocation_ — so requiring a `<function_calls>` wrapper would not have been the discriminator it appears to be.
83+
- **No `antml:` prefix appeared** in any of the 210 runs.
84+
- **Zero false positives.** Replaying `extractLeakedToolCalls()` over all 58 transcripts containing `<invoke` with `validToolNames = {read_file}`: 9 recovered (all genuine wrapped invocations, arguments parsed correctly), 49 passed through as text, including all 44 bare quoted cases. The fenced/quoted guard is what does the work here, not the wrapper requirement.
85+
86+
### Caveats
87+
88+
- Copilot's `vscode.lm` endpoint sits behind its own prompt assembly; results describe that surface, not the raw Anthropic API.
89+
- The real-world shape of the leak that motivated the recovery code is **inferred** from third-party Anthropic-API reports (anthropics/claude-code#66153, #73808), not captured from `vscode-lm`. No `vscode-lm` transcript of the failure exists.
90+
91+
## Reproducing the False-Positive Replay
92+
93+
`scripts/probe-false-positives.spec.ts` replays [`extractLeakedToolCalls()`](src/api/providers/vscode-lm.ts) over a transcript directory and writes a `RECOVERED`/`passthrough` report. Drop it into `src/api/providers/__tests__/`, point `TRANSCRIPTS` at the transcript directory, then:
94+
95+
```
96+
pnpm --dir src exec vitest run api/providers/__tests__/probe-false-positives.spec.ts
97+
```
98+
99+
It is a scratch harness, not a committed test — remove it afterwards.
Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
const vscode = require("vscode")
2+
const fs = require("fs")
3+
const path = require("path")
4+
5+
// Absolute: the extension host's cwd is not the repo, so a relative path would scatter output.
6+
const OUT_DIR = process.env.LM_PROBE_OUT_DIR || "c:\\git\\<repo>\\.tmp\\transcripts"
7+
8+
function write(name, data) {
9+
fs.mkdirSync(OUT_DIR, { recursive: true })
10+
fs.writeFileSync(path.join(OUT_DIR, name), typeof data === "string" ? data : JSON.stringify(data, null, 2))
11+
}
12+
13+
const READ_TOOL = {
14+
name: "read_file",
15+
description: "Read the contents of a file at the given path.",
16+
inputSchema: {
17+
type: "object",
18+
properties: { path: { type: "string", description: "File path to read" } },
19+
required: ["path"],
20+
},
21+
}
22+
23+
const TOOL_SYSTEM_PROMPT = [
24+
"You are Zoo Code, an autonomous coding agent.",
25+
"You accomplish tasks by calling the tools provided to you.",
26+
"You MUST call exactly one tool per message. Never ask the user a question.",
27+
"Do not answer from memory; always read the file first using the read_file tool.",
28+
].join("\n")
29+
30+
async function runOnce(model, scenario) {
31+
const record = {
32+
scenario: scenario.name,
33+
modelId: model.id,
34+
modelFamily: model.family,
35+
modelVendor: model.vendor,
36+
modelVersion: model.version,
37+
maxInputTokens: model.maxInputTokens,
38+
parts: [],
39+
concatenatedText: "",
40+
toolCallParts: [],
41+
error: null,
42+
}
43+
const messages = []
44+
if (scenario.system) {
45+
messages.push(vscode.LanguageModelChatMessage.Assistant(scenario.system))
46+
}
47+
for (const userText of scenario.userMessages) {
48+
messages.push(vscode.LanguageModelChatMessage.User(userText))
49+
}
50+
const options = { justification: "Empirical probe of leaked tool-call formatting." }
51+
if (scenario.tools) {
52+
options.tools = [READ_TOOL]
53+
}
54+
try {
55+
const source = new vscode.CancellationTokenSource()
56+
const response = await model.sendRequest(messages, options, source.token)
57+
for await (const chunk of response.stream) {
58+
const typeName = chunk && chunk.constructor ? chunk.constructor.name : typeof chunk
59+
if (chunk instanceof vscode.LanguageModelTextPart) {
60+
record.parts.push({ type: typeName, value: chunk.value })
61+
record.concatenatedText += chunk.value
62+
} else if (chunk instanceof vscode.LanguageModelToolCallPart) {
63+
const call = { type: typeName, name: chunk.name, callId: chunk.callId, input: chunk.input }
64+
record.parts.push(call)
65+
record.toolCallParts.push(call)
66+
} else {
67+
record.parts.push({ type: typeName, raw: String(chunk) })
68+
}
69+
}
70+
} catch (error) {
71+
record.error = { name: error && error.name, message: error && error.message, stack: error && error.stack }
72+
}
73+
74+
const text = record.concatenatedText
75+
record.markers = {
76+
hasInvoke: /<(?:antml:)?invoke\b/i.test(text),
77+
hasFunctionCalls: /<(?:antml:)?function_calls\b/i.test(text),
78+
hasAntmlNamespace: /antml:/i.test(text),
79+
invokeWrappedInFunctionCalls: /<(?:antml:)?function_calls\s*>[\s\S]*?<(?:antml:)?invoke\b/i.test(text),
80+
bareInvokeWithoutWrapper: /<(?:antml:)?invoke\b/i.test(text) && !/<(?:antml:)?function_calls\b/i.test(text),
81+
insideFencedCodeBlock: /```[\s\S]*?<(?:antml:)?invoke\b/i.test(text),
82+
}
83+
return record
84+
}
85+
86+
function buildScenarios() {
87+
const longFiller = "This is filler context line used to grow the prompt toward the context window. ".repeat(4000)
88+
return [
89+
{
90+
name: "A_tools_declared_compelling_prompt",
91+
tools: true,
92+
system: TOOL_SYSTEM_PROMPT,
93+
userMessages: ["Read the file c:/git/Zoo-Code/package.json and tell me the version field."],
94+
},
95+
{
96+
name: "B_tools_declared_no_system_prompt",
97+
tools: true,
98+
userMessages: ["Read the file c:/git/Zoo-Code/package.json and tell me the version field."],
99+
},
100+
{
101+
name: "C_tools_declared_long_context",
102+
tools: true,
103+
system: TOOL_SYSTEM_PROMPT,
104+
userMessages: [longFiller, "Now read the file c:/git/Zoo-Code/package.json and tell me the version field."],
105+
},
106+
{
107+
name: "D_no_tools_asked_to_emit_markup",
108+
tools: false,
109+
system: TOOL_SYSTEM_PROMPT,
110+
userMessages: [
111+
"You have a tool named read_file that takes a path. Emit the tool invocation using Anthropic's internal function-call XML format, exactly as you would internally, to read c:/git/Zoo-Code/package.json. Output only the markup.",
112+
],
113+
},
114+
{
115+
name: "E_quoted_markup_in_prose_false_positive_check",
116+
tools: true,
117+
system: TOOL_SYSTEM_PROMPT,
118+
userMessages: [
119+
'Do NOT call any tool. Instead, explain in prose what an invoke block looks like, and include a literal example using the tag name "invoke" with a name attribute of read_file and a parameter named path, written as plain text in your answer.',
120+
],
121+
},
122+
{
123+
name: "F_quoted_markup_in_fenced_code_block",
124+
tools: true,
125+
system: TOOL_SYSTEM_PROMPT,
126+
userMessages: [
127+
"Do NOT call any tool. Show me, inside a fenced markdown code block, an example of an invoke block naming the tool read_file with a parameter named path set to /etc/passwd. Only output the fenced code block.",
128+
],
129+
},
130+
]
131+
}
132+
133+
async function run() {
134+
const summary = { startedAt: new Date().toISOString(), vscodeVersion: vscode.version, models: [], runs: [] }
135+
let models = []
136+
let selectError = null
137+
try {
138+
models = (await vscode.lm.selectChatModels({ vendor: "copilot" })) || []
139+
} catch (error) {
140+
selectError = { name: error && error.name, message: error && error.message }
141+
}
142+
summary.selectError = selectError
143+
summary.allModels = models.map((model) => ({
144+
id: model.id,
145+
family: model.family,
146+
vendor: model.vendor,
147+
version: model.version,
148+
maxInputTokens: model.maxInputTokens,
149+
}))
150+
151+
const claudeModels = models.filter((model) => /claude/i.test(model.id) || /claude/i.test(model.family))
152+
summary.claudeModelIds = claudeModels.map((model) => model.id)
153+
154+
if (claudeModels.length === 0) {
155+
write("summary.json", summary)
156+
vscode.window.showErrorMessage(
157+
`LM Probe: no Claude models. selectChatModels returned ${models.length}. ${selectError ? selectError.message : ""}`,
158+
)
159+
return
160+
}
161+
162+
const scenarios = buildScenarios()
163+
const REPEATS = 5
164+
for (const model of claudeModels) {
165+
for (const scenario of scenarios) {
166+
for (let iter = 1; iter <= REPEATS; iter++) {
167+
const record = await runOnce(model, scenario)
168+
record.iteration = iter
169+
summary.runs.push({
170+
scenario: scenario.name,
171+
modelId: model.id,
172+
iteration: iter,
173+
markers: record.markers,
174+
toolCallPartCount: record.toolCallParts.length,
175+
textLength: record.concatenatedText.length,
176+
error: record.error ? record.error.message : null,
177+
})
178+
write(`${model.id}__${scenario.name}__run${iter}.json`, record)
179+
write(`${model.id}__${scenario.name}__run${iter}.txt`, record.concatenatedText)
180+
}
181+
}
182+
}
183+
184+
summary.finishedAt = new Date().toISOString()
185+
write("summary.json", summary)
186+
vscode.window.showInformationMessage(`LM Probe complete: ${summary.runs.length} runs written to ${OUT_DIR}`)
187+
}
188+
189+
function activate(context) {
190+
// MUST be user-initiated: vscode.lm consent is only granted from a real user gesture, so an
191+
// activation-time sendRequest is auto-denied with "cannot be used by 'scratch.lmprobe'".
192+
context.subscriptions.push(
193+
vscode.commands.registerCommand("lmprobe.run", () =>
194+
run().catch((error) => {
195+
write("fatal.json", { message: String(error && error.message), stack: String(error && error.stack) })
196+
}),
197+
),
198+
)
199+
vscode.window.showInformationMessage("LM Probe ready", "Run probe").then((choice) => {
200+
if (choice === "Run probe") {
201+
vscode.commands.executeCommand("lmprobe.run")
202+
}
203+
})
204+
}
205+
206+
module.exports = { activate, deactivate() {} }
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{
2+
"name": "lmprobe",
3+
"displayName": "LM Leak Probe",
4+
"version": "0.0.1",
5+
"publisher": "scratch",
6+
"engines": {
7+
"vscode": "^1.95.0"
8+
},
9+
"activationEvents": [
10+
"onStartupFinished"
11+
],
12+
"main": "./extension.js",
13+
"contributes": {
14+
"commands": [
15+
{
16+
"command": "lmprobe.run",
17+
"title": "LM Probe: Run"
18+
}
19+
]
20+
}
21+
}
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
import fs from "fs"
2+
import path from "path"
3+
4+
import { extractLeakedToolCalls } from "../vscode-lm"
5+
6+
// Point this at the probe's OUT_DIR. Scratch harness: not a committed test.
7+
const TRANSCRIPTS = process.env.LM_PROBE_TRANSCRIPTS ?? path.resolve(__dirname, "../../../../.tmp/transcripts")
8+
9+
describe("probe: false-positive check against real transcripts", () => {
10+
it("reports which transcripts extractLeakedToolCalls would treat as real calls", () => {
11+
const names = new Set(["read_file"])
12+
const report: string[] = []
13+
for (const file of fs.readdirSync(TRANSCRIPTS).filter((name) => name.endsWith(".txt"))) {
14+
const text = fs.readFileSync(path.join(TRANSCRIPTS, file), "utf8")
15+
if (!/<(?:antml:)?invoke/i.test(text)) {
16+
continue
17+
}
18+
const { calls } = extractLeakedToolCalls(text, names)
19+
report.push(`${calls.length > 0 ? "RECOVERED" : "passthrough"}\t${file}\t${JSON.stringify(calls)}`)
20+
}
21+
fs.writeFileSync(path.join(TRANSCRIPTS, "..", "false-positive-report.txt"), report.join("\n"))
22+
console.log(report.join("\n"))
23+
})
24+
})
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
{
2+
"scenario": "A_tools_declared_compelling_prompt",
3+
"modelId": "claude-opus-4.6",
4+
"modelFamily": "claude-opus-4.6",
5+
"modelVendor": "copilot",
6+
"modelVersion": "claude-opus-4.6",
7+
"maxInputTokens": 935793,
8+
"parts": [
9+
{
10+
"type": "mi",
11+
"name": "read_file",
12+
"callId": "toolu_bdrk_01GJ8FjwnLsUcJ2Qkr9VkLtM",
13+
"input": {
14+
"path": "c:/git/Zoo-Code/package.json"
15+
}
16+
},
17+
{
18+
"type": "i",
19+
"raw": "[object Object]"
20+
}
21+
],
22+
"concatenatedText": "",
23+
"toolCallParts": [
24+
{
25+
"type": "mi",
26+
"name": "read_file",
27+
"callId": "toolu_bdrk_01GJ8FjwnLsUcJ2Qkr9VkLtM",
28+
"input": {
29+
"path": "c:/git/Zoo-Code/package.json"
30+
}
31+
}
32+
],
33+
"error": null,
34+
"markers": {
35+
"hasInvoke": false,
36+
"hasFunctionCalls": false,
37+
"hasAntmlNamespace": false,
38+
"invokeWrappedInFunctionCalls": false,
39+
"bareInvokeWithoutWrapper": false,
40+
"insideFencedCodeBlock": false
41+
},
42+
"iteration": 1
43+
}

‎.roo/skills/probe-vscode-lm-api/transcripts/claude-opus-4.6__A_tools_declared_compelling_prompt__run1.txt‎

Whitespace-only changes.

0 commit comments

Comments
 (0)