Skip to content

Commit 20d2ef7

Browse files
feat(copilot): report host tool support, and release pre-send request state
Read each Copilot model's tool support from what VS Code reports, as Zoo-Code-Org#1971 requires. The resolver reads `capabilities.supportsToolCalling` (consumer side) or `toolCalling` (provider side, a boolean or a tool count) into a new optional ModelInfo.supportsToolCalling. A count only answers "supported?" and is never surfaced as a per-model tool limit. A model the host reports nothing about stays unset, so "not reported" is distinct from "unsupported", and the curated catalog's flag is not borrowed. The Copilot model details show "Supports tools", "Does not support tools", or "Tool support not reported"; other providers are unchanged. When token counting or a pre-send check fails, release the request's cancellation source at once instead of leaving it for the next request. Add a getGitHubCopilotAccount test for when no supported scope set holds a session.
1 parent a3f5133 commit 20d2ef7

24 files changed

Lines changed: 259 additions & 5 deletions

‎packages/types/src/model.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,9 @@ export const modelInfoSchema = z.object({
104104
maxThinkingTokens: z.number().nullish(),
105105
contextWindow: z.number(),
106106
supportsImages: z.boolean().optional(),
107+
// Whether the model accepts tool (function) calls, as its host reports it. Only hosts that
108+
// report this set it; unset means "not reported", not "unsupported".
109+
supportsToolCalling: z.boolean().optional(),
107110
supportsPromptCache: z.boolean(),
108111
// Optional default prompt cache retention policy for providers that support it.
109112
// When set to "24h", extended prompt caching will be requested; when omitted

‎src/api/providers/__tests__/github-copilot.spec.ts‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,14 @@ describe("connectGitHubCopilot", () => {
218218
expect(vscode.authentication.getSession).toHaveBeenCalledTimes(1)
219219
})
220220

221+
it("reports no account when no supported scope set holds a session", async () => {
222+
vi.mocked(vscode.authentication.getSession).mockResolvedValue(undefined)
223+
await expect(getGitHubCopilotAccount()).resolves.toBeUndefined()
224+
expect(vscode.authentication.getSession).toHaveBeenCalledTimes(2)
225+
expect(vscode.authentication.getSession).toHaveBeenCalledWith("github", ["user:email"], { silent: true })
226+
expect(vscode.authentication.getSession).toHaveBeenCalledWith("github", ["read:user"], { silent: true })
227+
})
228+
221229
it("reports no account rather than failing when the lookup throws", async () => {
222230
vi.mocked(vscode.authentication.getSession).mockRejectedValue(new Error("auth unavailable"))
223231
await expect(getGitHubCopilotAccount()).resolves.toBeUndefined()
@@ -437,6 +445,64 @@ describe("Copilot model capabilities", () => {
437445
})
438446
})
439447

448+
describe("tool support resolution", () => {
449+
it.each([true, false])("treats the host-reported tool flag (%s) as authoritative", (reported) => {
450+
const info = getVsCodeLmModelInfo(liveModel({ capabilities: { supportsToolCalling: reported } }))
451+
expect(info.supportsToolCalling).toBe(reported)
452+
})
453+
454+
it.each([
455+
[true, 128],
456+
[true, 1],
457+
[false, 0],
458+
])("reads the provider-side tool count as supported=%s for %s tools, never as a limit", (expected, count) => {
459+
const info = getVsCodeLmModelInfo(liveModel({ capabilities: { toolCalling: count } }))
460+
expect(info.supportsToolCalling).toBe(expected)
461+
// The count must not be carried into model info as if it were a per-model tool limit.
462+
const withoutCount = getVsCodeLmModelInfo(liveModel({ capabilities: { toolCalling: expected } }))
463+
expect(Object.keys(info).sort()).toEqual(Object.keys(withoutCount).sort())
464+
})
465+
466+
it("accepts the provider-side boolean spelling when the consumer-side one is absent", () => {
467+
expect(getVsCodeLmModelInfo(liveModel({ capabilities: { toolCalling: true } })).supportsToolCalling).toBe(
468+
true,
469+
)
470+
})
471+
472+
it("does not borrow the curated catalog's tool flag for a model the host reports nothing about", () => {
473+
// The curated catalog marks every family `supportsToolCalling: true`; that is not a host report.
474+
expect(getVsCodeLmModelInfo(liveModel({ family: curatedFamily })).supportsToolCalling).toBeUndefined()
475+
})
476+
477+
it.each([
478+
["no capabilities object", undefined],
479+
["an empty capabilities object", {}],
480+
["a capabilities object that says nothing about tools", { supportsImageToText: true }],
481+
["tools reported as something other than a boolean or count", { supportsToolCalling: "yes" }],
482+
["a non-finite tool count", { toolCalling: Number.NaN }],
483+
])("leaves tool support unset for %s", (_label, capabilities) => {
484+
expect(getVsCodeLmModelInfo(liveModel({ capabilities })).supportsToolCalling).toBeUndefined()
485+
})
486+
487+
it("keeps tool support independent of vision", () => {
488+
const info = getVsCodeLmModelInfo(
489+
liveModel({ capabilities: { supportsImageToText: false, supportsToolCalling: true } }),
490+
)
491+
expect(info.supportsImages).toBe(false)
492+
expect(info.supportsToolCalling).toBe(true)
493+
})
494+
495+
it("passes the host report through to the models Copilot lists", async () => {
496+
vi.mocked(vscode.lm.selectChatModels).mockResolvedValueOnce([
497+
liveModel({ id: "tools-yes", capabilities: { supportsToolCalling: true } }),
498+
liveModel({ id: "tools-unreported" }),
499+
] as never)
500+
const models = await getGitHubCopilotModels()
501+
expect(models.find((m) => m.id === "tools-yes")?.modelInfo.supportsToolCalling).toBe(true)
502+
expect(models.find((m) => m.id === "tools-unreported")?.modelInfo.supportsToolCalling).toBeUndefined()
503+
})
504+
})
505+
440506
describe("context window", () => {
441507
it("uses the live input limit when the catalog has no entry", () => {
442508
expect(getVsCodeLmModelInfo(liveModel({ maxInputTokens: 921793 })).contextWindow).toBe(921793)
@@ -693,6 +759,34 @@ describe("GitHubCopilotHandler", () => {
693759
expect(mockLanguageModelChat.sendRequest).not.toHaveBeenCalled()
694760
})
695761

762+
it("releases the request's cancellation source when a request is rejected before sending", async () => {
763+
// A pre-send check that throws without cleaning up after itself, so only the shared
764+
// request path can release the source. (Copilot's own context-window check also releases it.)
765+
const handler = new GitHubCopilotHandler(defaultOptions)
766+
handler["assertWithinContextWindow"] = () => {
767+
throw new Error("rejected before sending")
768+
}
769+
try {
770+
await expect(
771+
collectStream(handler.createMessage("system", [{ role: "user", content: "hi" }], { taskId: "t" })),
772+
).rejects.toThrow("rejected before sending")
773+
774+
// The request's source is the one token counting used; it must be released at failure,
775+
// not left for the next request's or the handler's cleanup.
776+
const countToken = mockLanguageModelChat.countTokens.mock.calls[0][1]
777+
const requestSource = vi
778+
.mocked(vscode.CancellationTokenSource)
779+
.mock.results.map((r) => r.value)
780+
.find((source) => source.token === countToken)
781+
expect(requestSource).toBeDefined()
782+
expect(requestSource.dispose).toHaveBeenCalled()
783+
expect(handler["currentRequestCancellation"]).toBeNull()
784+
expect(mockLanguageModelChat.sendRequest).not.toHaveBeenCalled()
785+
} finally {
786+
handler.dispose()
787+
}
788+
})
789+
696790
it("cancels before sending when the task aborts during token counting", async () => {
697791
const controller = new AbortController()
698792
mockLanguageModelChat.countTokens.mockImplementation(async () => {

‎src/api/providers/vscode-lm-capabilities.ts‎

Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,12 @@ import {
99

1010
import { canCreateImageParts } from "../transform/vscode-lm-image-part"
1111

12+
/** The model's `capabilities` object, read by name because the stable typings do not declare it. */
13+
function reportedCapabilities(model: vscode.LanguageModelChat): object | undefined {
14+
const capabilities: unknown = Reflect.get(model, "capabilities")
15+
return typeof capabilities === "object" && capabilities !== null ? capabilities : undefined
16+
}
17+
1218
/**
1319
* Vision as the host reports it.
1420
*
@@ -18,8 +24,8 @@ import { canCreateImageParts } from "../transform/vscode-lm-image-part"
1824
* so the caller can tell "unsupported" apart from "unknown".
1925
*/
2026
export function reportedImageSupport(model: vscode.LanguageModelChat): boolean | undefined {
21-
const capabilities: unknown = Reflect.get(model, "capabilities")
22-
if (typeof capabilities !== "object" || capabilities === null) {
27+
const capabilities = reportedCapabilities(model)
28+
if (!capabilities) {
2329
return undefined
2430
}
2531
for (const key of ["supportsImageToText", "imageInput"]) {
@@ -31,6 +37,32 @@ export function reportedImageSupport(model: vscode.LanguageModelChat): boolean |
3137
return undefined
3238
}
3339

40+
/**
41+
* Tool calling as the host reports it.
42+
*
43+
* VS Code exposes this to consumers as `capabilities.supportsToolCalling` (same proposed surface as
44+
* vision) and to providers as `capabilities.toolCalling`, which may be a boolean or the number of tools
45+
* the model accepts. A number only answers "supported?" here: positive means yes, zero means no. Zoo
46+
* deliberately does not surface it as a per-model tool limit. Anything else yields `undefined`, so
47+
* "not reported" stays distinct from "unsupported".
48+
*/
49+
export function reportedToolSupport(model: vscode.LanguageModelChat): boolean | undefined {
50+
const capabilities = reportedCapabilities(model)
51+
if (!capabilities) {
52+
return undefined
53+
}
54+
for (const key of ["supportsToolCalling", "toolCalling"]) {
55+
const reported: unknown = Reflect.get(capabilities, key)
56+
if (typeof reported === "boolean") {
57+
return reported
58+
}
59+
if (typeof reported === "number" && Number.isFinite(reported)) {
60+
return reported > 0
61+
}
62+
}
63+
return undefined
64+
}
65+
3466
/**
3567
* Derives Zoo's model info from what VS Code reports for a live model. The curated catalog only fills
3668
* gaps in everything but vision, and anything neither source states stays unset rather than guessed.
@@ -50,6 +82,8 @@ export function getVsCodeLmModelInfo(model: vscode.LanguageModelChat): ModelInfo
5082
curatedInputLimit && curatedInputLimit > 0 ? Math.min(liveInputLimit, curatedInputLimit) : liveInputLimit,
5183
// A live model's vision is the host's word alone; unreported stays unset rather than borrowing the catalog's.
5284
supportsImages: canCreateImageParts() ? reportedImageSupport(model) : false,
85+
// Likewise tool calling: the curated catalog's `supportsToolCalling` is not a host report.
86+
supportsToolCalling: reportedToolSupport(model),
5387
supportsPromptCache: false,
5488
inputPrice: 0,
5589
outputPrice: 0,

‎src/api/providers/vscode-lm.ts‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1197,9 +1197,17 @@ export class VsCodeLmHandler extends BaseProvider implements SingleCompletionHan
11971197
const abortSignal = metadata?.abortSignal
11981198
const cancelOnAbort = () => requestCancellation.cancel()
11991199

1200-
// Calculate input tokens before starting the stream
1201-
const totalInputTokens: number = await this.calculateTotalInputTokens(vsCodeLmMessages)
1202-
this.assertWithinContextWindow(totalInputTokens, modelInfo)
1200+
// Calculate input tokens before starting the stream. A failure here ends the request
1201+
// before it is sent, so release its cancellation source now instead of leaving it
1202+
// for the next request's ensureCleanState(). The abort listener is not attached yet.
1203+
let totalInputTokens: number
1204+
try {
1205+
totalInputTokens = await this.calculateTotalInputTokens(vsCodeLmMessages)
1206+
this.assertWithinContextWindow(totalInputTokens, modelInfo)
1207+
} catch (error) {
1208+
this.ensureCleanState()
1209+
throw error
1210+
}
12031211

12041212
// Accumulate the text and count at the end of the stream to reduce token counting overhead.
12051213
let accumulatedText: string = ""

‎webview-ui/src/components/settings/ModelInfoView.tsx‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,21 @@ export const ModelInfoView = ({
8282
doesNotSupportLabel={t("settings:modelInfo.noImages")}
8383
/>
8484
),
85+
// Tool support is shown only where the host reports it per model. It is a yes/no answer:
86+
// VS Code does not report how many tools a model accepts, so no limit is implied.
87+
apiProvider === providerIdentifiers.githubCopilot &&
88+
(modelInfo?.supportsToolCalling === undefined ? (
89+
<div className="flex items-center gap-1 font-medium">
90+
<span className="codicon codicon-info" aria-hidden="true" />
91+
{t("settings:modelInfo.toolSupportUnknown")}
92+
</div>
93+
) : (
94+
<ModelInfoSupportsItem
95+
isSupported={modelInfo.supportsToolCalling}
96+
supportsLabel={t("settings:modelInfo.supportsTools")}
97+
doesNotSupportLabel={t("settings:modelInfo.noTools")}
98+
/>
99+
)),
85100
apiProvider === providerIdentifiers.githubCopilot ? (
86101
<StandardTooltip content={t("settings:modelInfo.promptCacheCopilotDetails")}>
87102
<div

‎webview-ui/src/components/settings/__tests__/ModelInfoView.spec.tsx‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@ vi.mock("@src/i18n/TranslationContext", () => ({
1111
"settings:modelInfo.imageSupportUnknown": "Image support not reported",
1212
"settings:modelInfo.supportsImages": "Supports images",
1313
"settings:modelInfo.noImages": "Does not support images",
14+
"settings:modelInfo.toolSupportUnknown": "Tool support not reported",
15+
"settings:modelInfo.supportsTools": "Supports tools",
16+
"settings:modelInfo.noTools": "Does not support tools",
1417
"settings:modelInfo.noPromptCache": "Does not support prompt caching",
1518
"settings:modelInfo.promptCacheManagedByCopilot": "Prompt caching managed by GitHub Copilot",
1619
"settings:serviceTier.pricingTableTitle": "Service tier pricing",
@@ -103,6 +106,49 @@ describe("ModelInfoView service tier pricing", () => {
103106
expect(screen.queryByText("Does not support prompt caching")).not.toBeInTheDocument()
104107
})
105108

109+
it.each([
110+
[true, "Supports tools"],
111+
[false, "Does not support tools"],
112+
])("shows host-reported Copilot tool support (%s) as reported", (supportsToolCalling, label) => {
113+
render(
114+
<ModelInfoView
115+
{...defaultProps}
116+
apiProvider={providerIdentifiers.githubCopilot}
117+
modelInfo={{ contextWindow: 128000, supportsPromptCache: false, supportsToolCalling }}
118+
hidePricing
119+
/>,
120+
)
121+
expect(screen.getByText(label)).toBeInTheDocument()
122+
expect(screen.queryByText("Tool support not reported")).not.toBeInTheDocument()
123+
})
124+
125+
it("shows unreported Copilot tool support without claiming either way", () => {
126+
render(
127+
<ModelInfoView
128+
{...defaultProps}
129+
apiProvider={providerIdentifiers.githubCopilot}
130+
modelInfo={{ contextWindow: 128000, supportsPromptCache: false }}
131+
hidePricing
132+
/>,
133+
)
134+
expect(screen.getByText("Tool support not reported")).toBeInTheDocument()
135+
expect(screen.queryByText("Supports tools")).not.toBeInTheDocument()
136+
expect(screen.queryByText("Does not support tools")).not.toBeInTheDocument()
137+
})
138+
139+
it("does not add a tool-support row for other providers", () => {
140+
render(
141+
<ModelInfoView
142+
{...defaultProps}
143+
apiProvider={providerIdentifiers.openai}
144+
modelInfo={{ contextWindow: 128000, supportsPromptCache: false, supportsToolCalling: true }}
145+
hidePricing
146+
/>,
147+
)
148+
expect(screen.queryByText("Supports tools")).not.toBeInTheDocument()
149+
expect(screen.queryByText("Tool support not reported")).not.toBeInTheDocument()
150+
})
151+
106152
it("keeps unsupported caching labels for providers whose models do not support caching", () => {
107153
render(
108154
<ModelInfoView

‎webview-ui/src/i18n/locales/ca/settings.json‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎webview-ui/src/i18n/locales/de/settings.json‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎webview-ui/src/i18n/locales/en/settings.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1093,10 +1093,13 @@
10931093
},
10941094
"modelInfo": {
10951095
"imageSupportUnknown": "Image support not reported",
1096+
"toolSupportUnknown": "Tool support not reported",
10961097
"promptCacheManagedByCopilot": "Prompt caching managed by GitHub Copilot",
10971098
"promptCacheCopilotDetails": "Caching depends on the model and GitHub Copilot backend. Zoo Code cannot configure cache retention or observe cache hits through the VS Code API.",
10981099
"supportsImages": "Supports images",
10991100
"noImages": "Does not support images",
1101+
"supportsTools": "Supports tools",
1102+
"noTools": "Does not support tools",
11001103
"supportsPromptCache": "Supports prompt caching",
11011104
"noPromptCache": "Does not support prompt caching",
11021105
"contextWindow": "Context Window:",

‎webview-ui/src/i18n/locales/es/settings.json‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)