Skip to content

Commit 14332f2

Browse files
Fix Foundry Local agent reliability + telemetry pipeline
The /api/command endpoint was reliably returning RUN_ERROR for all chats because of three stacked Foundry Local + agent-framework bugs. This commit fixes all three plus a telemetry pipeline bug discovered during verification, plus two minor production papercuts found along the way. ## Streaming wrapper (agent.py) Added _NormalizingChatClient._inner_get_response that translates AG-UI's forced stream=True into a non-streaming Foundry call, then synthesizes a single ChatCompletionChunk so the rest of agent-framework sees what it expects. Foundry-Local#422 makes streaming unreliable inside the long-running uvicorn process; non-streaming works. Also strips AG-UI's auto-injected "Current state of the application:" system message before sending to Foundry. With 200+ todos this message is 75k+ chars and blows the 32k context window of qwen2.5-coder-0.5b. ## Smoke test (main.py) Added _smoke_test_inference that probes the FULL agent payload (live _build_instructions() + serialized ALL_TOOLS) before declaring a model switch successful. Catches Foundry NPU runtime crashes (#506) at switch time so users see a clear rollback instead of the server dying on the first chat. _commit_switch now snapshots prior state and restores on failure. Persists target_model.id (full ID, not alias) so switches survive restart and don't bounce back to broken NPU defaults. ## EP-aware startup (agent.py) Captures EpDownloadResult and filters cached models by registered execution providers. CPU + WebGPU are always considered registered. Without this, cached[0] auto-selection lands on a NPU model when VitisAI EP failed to register, leaving the agent unusable. ## Idempotent telemetry init (telemetry.py) Uvicorn imports main.py twice (worker + main process). OpenTelemetry refuses to replace the active TracerProvider, but the old code still mutated the module-level _ui_span_exporter on every call — leaving it pointing at an orphaned exporter that never received spans. Added an _otel_initialized guard so init_otel() is no-op on subsequent calls. This restored ai.response, flow, and http.request events in the Stats panel along with token counts. ## App config timezone shadow (app_config.py) The local timezone() function shadowed the datetime.timezone import, breaking app_config.set() with "'function' object has no attribute 'utc'". Imported datetime.timezone as _tz instead. ## Launch script Get-FileHash fallback (launch.ps1) On rare PS 5.1 configurations Microsoft.PowerShell.Utility isn't auto-imported when launch.ps1 runs via the taskbean:// protocol handler, surfacing a confusing red error. Added a manual SHA256 fallback so dep-stamp gating still works silently. ## Testing - 17 new unit tests in app/agent/test_phase0_fixes.py covering all five fixes including edge cases (size guard on state-context filter, smoke test rollback, persist-on-success, clear-on-failure). - Full Python suite: 113/113 (test_phase0_fixes + test_agui_refactor + test_version_endpoint + test_fs_endpoints + test_projects_api + test_usage_scanners). - CLI suite: 149/149. - Manual E2E verified end-to-end through the UI: chat works, tasks created, ai.response/flow/http.request spans appear in the Stats panel with token counts and durations. ## Known limitation: NPU + agent payloads NPU runtimes (VitisAI, QNN) reliably crash Foundry on requests containing the full system prompt + 10-tool definition list. This is Foundry-Local#506 and is out of taskbean's control. The smoke test catches this at switch time and rolls back with an actionable error message ("Foundry closed the connection ... NPU models are particularly prone to this — see Foundry-Local#506") rather than letting the real chat take down the server. CPU and GPU paths are unaffected. ## Documentation Updated .github/copilot-instructions.md with the Playwright MCP recovery procedure (use configure-copilot agent to restart the MCP server, then click the bridge extension to reconnect). References: - microsoft/foundry-local#422 (streaming bug) - microsoft/foundry-local#506 (NPU + tools) - microsoft/foundry-local#558 (VitisAI EP) - microsoft/agent-framework#4084 (multi-modal content) Co-authored-by: Copilot <[email protected]>
1 parent 1da3503 commit 14332f2

7 files changed

Lines changed: 964 additions & 13 deletions

File tree

‎.github/copilot-instructions.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -306,6 +306,29 @@ The project has a Playwright MCP server configured in `.mcp.json` using `--exten
306306
4. **Never use screenshots for element targeting.** `browser_take_screenshot` is for visual verification only — it does not provide actionable refs. The snapshot tool description says: *"this is better than screenshot."*
307307
5. **Use `element` descriptions for clarity.** When calling action tools, fill in the `element` parameter with a human-readable description (e.g., "the Send button") alongside the `ref`.
308308

309+
### Recovering from stale connections
310+
311+
The Playwright MCP uses `--extension` mode, which connects to Edge via a WebSocket from the browser's MCP Bridge extension. This connection can break when:
312+
313+
- Edge crashes or is closed and reopened
314+
- The Python backend crashes mid-request (causes page navigation to fail, which can kill the bridge's page reference)
315+
- The MCP server process caches a dead browser/page reference and never recovers on its own
316+
317+
**Symptoms:** Every MCP tool call returns `Error: browserBackend.callTool: Target page, context or browser has been closed`.
318+
319+
**Fix — restart the Playwright MCP server** so it establishes a fresh WebSocket to the bridge extension:
320+
321+
1. Use the `configure-copilot` **task agent** (`agent_type: "configure-copilot"`) to reload the MCP server. This restarts the server process without changing config. Ask it to "reload/restart the playwright MCP server" and specify not to change configuration.
322+
2. After restart, the bridge extension needs to reconnect. The error changes from `"Target page, context or browser has been closed"` to `"Not connected"` — this confirms the restart worked. You **cannot** navigate to `extension://mmlmfjhmonkocbjadbfplnigmagldckm/status.html` yourself because all browser tools are dead until the extension connects. Ask the user to click the Playwright MCP Bridge extension icon in Edge to re-establish the WebSocket.
323+
3. If that doesn't work, ask the user to run `/mcp` and restart the `playwright` server manually, or restart Copilot CLI entirely.
324+
4. After the extension connects, verify with `browser_snapshot` — you should see the extension's connect/status page, then navigate to `http://localhost:8275`.
325+
326+
**Do NOT** try to work around the stale connection by:
327+
- Launching a separate browser via `playwright-cli open` (creates a second browser instance disconnected from the MCP tools)
328+
- Repeatedly retrying `browser_navigate` (the cached reference never self-heals)
329+
- Killing individual node processes (the MCP server is managed by Copilot CLI's process lifecycle)
330+
- Using the `customize-cloud-agent` skill (that's for the cloud coding agent CI/CD, not local MCP servers)
331+
309332
### App-specific notes
310333

311334
- The app runs on **`http://localhost:8275`** — the Python backend must be running first (`cd app/agent && python main.py`).

‎app/agent/agent.py‎

Lines changed: 228 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,18 @@ def _update_model_context(model: Any) -> None:
6060
)
6161

6262

63+
def _model_ep(model: Any) -> str:
64+
"""Return the execution provider a model requires, e.g.
65+
``CPUExecutionProvider`` or ``VitisAIExecutionProvider``.
66+
67+
Returns the empty string if the EP can't be determined; callers should
68+
treat that as "skip this model" since they can't know if it's safe.
69+
"""
70+
info = getattr(model, "info", None)
71+
runtime = getattr(info, "runtime", None) if info else None
72+
return getattr(runtime, "execution_provider", "") if runtime else ""
73+
74+
6375
# ── Initialization ────────────────────────────────────────────────────────────
6476

6577
async def _ensure_web_service() -> None:
@@ -109,8 +121,20 @@ async def initialize_foundry(
109121

110122
# Download and register hardware execution providers (VITIS/QNN for NPU,
111123
# CUDA/TensorRT for GPU, etc.). This is a no-op when already registered.
124+
# Capture the result so we can filter cached models below.
112125
logger.info("Registering execution providers…")
113126
await asyncio.to_thread(manager.download_and_register_eps)
127+
# Build the set of EPs that are actually usable. CPU and WebGPU are
128+
# always built in. discover_eps returns explicit EPs (VitisAI, QNN,
129+
# MIGraphX, TensorRT, etc.) with their current registration state.
130+
registered_eps = {"CPUExecutionProvider", "WebGpuExecutionProvider"}
131+
try:
132+
for ep in await asyncio.to_thread(manager.discover_eps):
133+
if ep.is_registered:
134+
registered_eps.add(ep.name)
135+
except Exception as exc:
136+
logger.warning("Failed to enumerate execution providers: %s", exc)
137+
logger.info("Registered execution providers: %s", sorted(registered_eps))
114138
foundry_ready = True
115139
logger.info("Foundry Local SDK initialized")
116140

@@ -140,13 +164,43 @@ async def initialize_foundry(
140164
pass
141165

142166
if model is None:
143-
# No explicit preference — pick the best cached (downloaded) model.
167+
# No explicit preference — pick the best cached (downloaded) model
168+
# whose execution provider is actually registered. Without this
169+
# filter, cached[0] can land on an NPU model when the VitisAI EP
170+
# failed to register, leaving the agent unusable until manual
171+
# intervention. EP filter is a necessary first cut; the smoke
172+
# test in /api/models/switch is the real readiness check.
144173
cached = await asyncio.to_thread(
145174
lambda: list(manager.catalog.get_cached_models())
146175
)
147-
if cached:
148-
model = cached[0]
149-
logger.info("Auto-selected cached model: %s", model.alias)
176+
compatible = [
177+
m for m in cached
178+
if _model_ep(m) in registered_eps
179+
]
180+
if compatible:
181+
model = compatible[0]
182+
logger.info(
183+
"Auto-selected cached model: %s (ep=%s)",
184+
model.alias, _model_ep(model),
185+
)
186+
if len(compatible) < len(cached):
187+
skipped = [m.alias for m in cached if m not in compatible]
188+
logger.info(
189+
"Skipped %d cached model(s) with unregistered EPs: %s",
190+
len(skipped), skipped,
191+
)
192+
elif cached:
193+
# Cached models exist but none are EP-compatible — log a
194+
# clear warning so the user knows to restore the EP (e.g.
195+
# via Foundry-Local#558 PowerShell cleanup) or download a
196+
# CPU/GPU model.
197+
ep_summary = sorted({_model_ep(m) for m in cached})
198+
raise RuntimeError(
199+
f"No cached models compatible with registered EPs "
200+
f"{sorted(registered_eps)}. Cached models require: {ep_summary}. "
201+
"Either restore the missing execution provider or download "
202+
"a CPU/GPU model variant."
203+
)
150204
else:
151205
raise RuntimeError(
152206
"No cached models available. Download a model first: "
@@ -229,6 +283,18 @@ class _NormalizingChatClient(OpenAIChatCompletionClient):
229283
230284
For models that already return native ``tool_calls`` the XML regex won't
231285
match, making this a transparent no-op.
286+
287+
**Known limitation: NPU models often crash Foundry on agent payloads.**
288+
Foundry NPU runtimes (VitisAI, QNN) crash when handed a request with the
289+
full system prompt + 10-tool definition list — see
290+
https://github.com/microsoft/Foundry-Local/issues/506. The crash kills
291+
Foundry's HTTP server, which in turn drops our connection and causes
292+
httpx.RemoteProtocolError. The `_smoke_test_inference` in main.py probes
293+
this BEFORE declaring a model switch successful, using the exact same
294+
payload shape (full instructions + ALL_TOOLS). NPU models that can't
295+
handle the agent payload get rolled back at switch time with a clear
296+
error message instead of silently dying on the first chat. CPU and GPU
297+
models are unaffected.
232298
"""
233299

234300
def _parse_tool_calls_from_openai(self, choice): # type: ignore[override]
@@ -268,6 +334,164 @@ def _parse_text_from_openai(self, choice): # type: ignore[override]
268334
return Content.from_text(text=stripped, raw_representation=choice) if stripped else None
269335
return super()._parse_text_from_openai(choice)
270336

337+
def _inner_get_response( # type: ignore[override]
338+
self,
339+
*,
340+
messages,
341+
options,
342+
stream: bool = False,
343+
**kwargs,
344+
):
345+
"""Translate streaming requests into a non-streaming Foundry call.
346+
347+
Foundry Local's ``/v1/chat/completions`` with ``stream=true`` reliably
348+
drops the connection mid-response when called from the long-running
349+
FastAPI/uvicorn server (httpx surfaces this as
350+
``RemoteProtocolError: peer closed connection``). The non-streaming
351+
path works perfectly. We can't disable streaming at the AG-UI layer
352+
(``run_agent_stream`` always passes ``stream=True``), so we translate
353+
it here: when streaming is requested, we call Foundry with
354+
``stream=False``, then synthesize a single ``ChatCompletionChunk``
355+
that wraps the full response and feed it through the parent's
356+
``_parse_response_update_from_openai`` so the rest of the framework
357+
sees a normal-looking stream of one chunk.
358+
359+
Trade-off: the user sees the response appear all at once after
360+
generation completes, instead of token-by-token. This is acceptable
361+
for a tool-calling agent (where most "tokens" are JSON arguments
362+
that aren't shown to the user anyway) and unblocks all chat
363+
functionality.
364+
365+
Couples us to ``agent_framework_openai._chat_completion_client``
366+
internals (``_inner_get_response``, ``_parse_response_update_from_openai``,
367+
``_build_response_stream``). Pin the framework version. Revisit when
368+
Foundry-Local#422 is fixed and remove this override if so.
369+
"""
370+
# Strip the AG-UI auto-injected state-context system message before
371+
# ANY downstream processing. AG-UI's _create_state_context_message
372+
# dumps the full client state dict (in our case ~75k chars of todo
373+
# JSON) as a system message on every turn. The model already has a
374+
# compact view via _build_instructions(), and tools mutate state
375+
# directly, so the injection is redundant — and on small-context
376+
# models (qwen2.5-coder-0.5b: 32k tokens) it blows the context
377+
# window and Foundry returns HTTP 500.
378+
messages = [m for m in messages if not _is_agui_state_context(m)]
379+
380+
if not stream:
381+
return super()._inner_get_response(
382+
messages=messages, options=options, stream=False, **kwargs
383+
)
384+
385+
# Streaming requested — fold to non-streaming and synthesize one
386+
# chunk so the rest of agent-framework sees what it expects.
387+
# Trade-off: we bypass the parent's ChatTelemetryLayer wrapping
388+
# (chat <model> spans), so AI response telemetry is reduced for
389+
# this path. The lower-level FunctionInvocationLayer telemetry
390+
# (tool execution spans) still fires. Future work: wrap our raw
391+
# call in the same span so the nerd panel sees AI events too.
392+
async def _fake_stream():
393+
options_dict = self._prepare_options(messages, options)
394+
options_dict.pop("stream_options", None)
395+
try:
396+
completion = await self.client.chat.completions.create(
397+
stream=False, **options_dict
398+
)
399+
except Exception as ex:
400+
from agent_framework.exceptions import ChatClientException
401+
if isinstance(ex, ChatClientException):
402+
raise
403+
raise ChatClientException(
404+
f"{type(self)} service failed to complete the prompt: {ex}",
405+
inner_exception=ex,
406+
) from ex
407+
chunk = _completion_to_chunk(completion)
408+
yield self._parse_response_update_from_openai(chunk)
409+
410+
return self._build_response_stream(
411+
_fake_stream(), response_format=options.get("response_format")
412+
)
413+
414+
415+
def _is_agui_state_context(message) -> bool:
416+
"""Detect AG-UI's auto-injected ``Current state of the application:``
417+
system message so we can strip it before sending to Foundry.
418+
419+
Belt-and-suspenders: the prefix is unique to AG-UI's
420+
``_create_state_context_message`` helper so the prefix check alone is
421+
very unlikely to false-positive. We also require the message be
422+
suspiciously large (> 1 KB) to make absolutely sure we never strip a
423+
legitimate user-or-app-authored system message that happens to start
424+
with the same words. AG-UI's real injection is typically 75k+ chars
425+
(full state JSON), so 1 KB is a comfortably low bar.
426+
"""
427+
if getattr(message, "role", None) != "system":
428+
return False
429+
for content in getattr(message, "contents", None) or []:
430+
text = getattr(content, "text", None)
431+
if (
432+
isinstance(text, str)
433+
and text.startswith("Current state of the application:")
434+
and len(text) > 1024
435+
):
436+
return True
437+
return False
438+
439+
440+
def _completion_to_chunk(completion):
441+
"""Convert a non-streaming ``ChatCompletion`` to a ``ChatCompletionChunk``.
442+
443+
The chunk preserves all fields the agent-framework parser cares about:
444+
role, content, tool_calls, finish_reason, and usage. Used by
445+
``_NormalizingChatClient._inner_get_response`` to make a non-streaming
446+
response look like a one-event stream.
447+
"""
448+
from openai.types.chat import ChatCompletionChunk
449+
from openai.types.chat.chat_completion_chunk import (
450+
Choice,
451+
ChoiceDelta,
452+
ChoiceDeltaToolCall,
453+
ChoiceDeltaToolCallFunction,
454+
)
455+
456+
chunk_choices = []
457+
for choice in completion.choices:
458+
msg = choice.message
459+
delta_tool_calls = None
460+
if msg.tool_calls:
461+
delta_tool_calls = [
462+
ChoiceDeltaToolCall(
463+
index=i,
464+
id=tc.id,
465+
type=tc.type,
466+
function=ChoiceDeltaToolCallFunction(
467+
name=tc.function.name,
468+
arguments=tc.function.arguments,
469+
),
470+
)
471+
for i, tc in enumerate(msg.tool_calls)
472+
]
473+
delta = ChoiceDelta(
474+
role=msg.role,
475+
content=msg.content,
476+
tool_calls=delta_tool_calls,
477+
)
478+
chunk_choices.append(
479+
Choice(
480+
index=choice.index,
481+
delta=delta,
482+
finish_reason=choice.finish_reason,
483+
)
484+
)
485+
486+
return ChatCompletionChunk(
487+
id=completion.id,
488+
choices=chunk_choices,
489+
created=completion.created,
490+
model=completion.model,
491+
object="chat.completion.chunk",
492+
usage=completion.usage,
493+
)
494+
271495

272496
# Singleton agent instances — reassigned on model switch.
273497
_agent_singleton: Agent | None = None

‎app/agent/app_config.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
import asyncio
2525
import json
2626
import logging
27-
from datetime import datetime, timezone
27+
from datetime import datetime, timezone as _tz
2828
from pathlib import Path
2929
from typing import Literal
3030

@@ -106,7 +106,7 @@ def save() -> None:
106106
"""Persist current in-memory config to disk."""
107107
try:
108108
_CONFIG_DIR.mkdir(parents=True, exist_ok=True)
109-
out = {**_config, "lastUpdated": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")}
109+
out = {**_config, "lastUpdated": datetime.now(_tz.utc).isoformat().replace("+00:00", "Z")}
110110
_CONFIG_FILE.write_text(json.dumps(out, indent=2), encoding="utf-8")
111111
except Exception as exc:
112112
logger.warning("Could not save config: %s", exc)

0 commit comments

Comments
 (0)