Add experimental CodeMode transform by aaazzam · Pull Request #3297 · PrefectHQ/fastmcp

aaazzam · 2026-02-25T20:06:42Z

Summary

Adds CodeMode, an experimental transform inspired by Cloudflare's Code Mode that collapses a large tool surface into two meta-tools: search (inspect tool metadata via Python) and execute (chain call_tool(...) invocations in Python). Useful when LLMs need to plan multi-step workflows against many tools without listing them all in context.

Changes

New fastmcp.experimental.transforms.CodeMode transform with pluggable SandboxProvider protocol
Default MontySandboxProvider backed by pydantic-monty (new fastmcp[code-mode] optional dep)
Supports custom tool names/descriptions, default arguments, search helpers, and respects disabled/auth-gated tools
Documentation at docs/servers/transforms/code-mode.mdx
10 test cases with an in-process sandbox provider

Design notes

Placed under fastmcp.experimental.transforms since this is an experimental API. The sandbox provider is a protocol so third-party runtimes (remote sandboxes, WASM, etc.) can plug in without changes to the transform itself.

JL notes update from review:

replaced code-based search (from CloudFlare blog) with new search transforms from feat: Search transforms for tool discovery #3154. Way better performance and deterministic outcomes

marvin-context-protocol · 2026-02-25T20:11:38Z

Test Failure Analysis

Summary: The Windows CI job failed because test_server_performance_no_latency asserts that OpenAPIProvider initialization completes in under 100ms, but the Windows runner took ~317ms (~3x the threshold).

Root Cause: The test at tests/server/providers/openapi/test_comprehensive.py:736 uses a hard wall-clock timing assertion (< 0.1 seconds). Windows CI runners are typically slower than Linux, making this threshold unreliable across platforms.

Suggested Solution:

In tests/server/providers/openapi/test_comprehensive.py, either:

Increase the threshold to something reasonable for all platforms (e.g., < 1.0):
```
assert initialization_time < 1.0  # Should be well under 1 second
```

Skip on Windows (if Linux-only timing is sufficient):

@pytest.mark.skipif(sys.platform == "win32", reason="performance thresholds unreliable on Windows CI")
async def test_server_performance_no_latency(self, ...):

Option 1 is simpler and still validates the basic intent (no multi-second startup cost).

Detailed Analysis

Failed job: "Tests: Python 3.10 on windows-latest" (Job ID: 65171185865)

Log excerpt:

FAILED tests/server/providers/openapi/test_comprehensive.py::TestOpenAPIComprehensive::test_server_performance_no_latency
E       assert 0.3173520565032959 < 0.1
tests\server\providers\openapi\test_comprehensive.py:736: AssertionError

The test passed on:

Python 3.10 on ubuntu-latest ✅
Python 3.13 on ubuntu-latest ✅

But failed on:

Python 3.10 on windows-latest ❌ (317ms vs 100ms threshold)

This is a performance assertion that's sensitive to CI runner speed. The 100ms threshold is reasonable for a development machine but too tight for Windows CI runners, which are known to be slower than Linux runners.

All other 4,415 tests passed. This appears to be a pre-existing flakiness issue introduced with the test itself, not caused by the CodeMode transform changes.

Related Files

tests/server/providers/openapi/test_comprehensive.py (line 736) - Contains the failing performance assertion
src/fastmcp/server/providers/openapi/ - The OpenAPIProvider being tested

🤖 Automated triage via Claude Code · edited to reflect latest analysis

🤖 Generated with Claude Code

🤖 Generated with Claude Code Co-Authored-By: Claude Opus 4.6 <[email protected]>

Removes the duplicate ContextVar bypass pattern in favor of the shared CatalogTransform machinery. Also fixes a pre-existing bug where `from __future__ import annotations` caused NameError for Annotated in nested function scopes at runtime.

call_tool() inside execute blocks now returns structured content as-is, preserving the {"result": value} wrapping. This means the output schema shown in search results accurately describes what call_tool() returns, so LLMs can trust the schema when writing code. Also adds examples/code_mode/ with a server and narrated client demo.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d1b4c50977

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-27T16:55:41Z

+
+            async def call_tool(tool_name: str, params: dict[str, Any]) -> Any:
+                backend_tools = await _get_cached_tools()
+                tool = transform._find_tool(tool_name, backend_tools)


Select highest-version backend tool before execute call

execute resolves a backend tool by scanning the list_tools() catalog and taking the first name match, then pins that exact version for the call. Because list_tools() returns all versions in provider/registration order, a versioned tool like foo@1 + foo@2 can be invoked as foo@1 through CodeMode even though normal call_tool("foo", ...) semantics return the highest version; this causes silent behavior drift for any server that relies on versioned tools.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a296d4e895

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

marvin-context-protocol Bot added feature Major new functionality. Reserved for 2-4 significant PRs per release. Not for issues. server Related to FastMCP server implementation or server-side functionality. labels Feb 25, 2026

mintlify Bot deployed to staging - docs February 25, 2026 20:07 View deployment

This comment was marked as resolved.

Sign in to view

mintlify Bot deployed to staging - docs February 25, 2026 21:49 View deployment

mintlify Bot deployed to staging - docs February 25, 2026 23:13 View deployment

aaazzam and others added 13 commits February 26, 2026 22:43

code mode

aa6346c

update uv.lock for monty optional dep

a53ff4f

🤖 Generated with Claude Code

retry CI

14b239d

Address PR review comments on CodeMode transform

8c42ea5

🤖 Generated with Claude Code Co-Authored-By: Claude Opus 4.6 <[email protected]>

Fix ty unresolved-attribute error on search_helper

1507527

🤖 Generated with Claude Code Co-Authored-By: Claude Opus 4.6 <[email protected]>

more idiomacy

16f3fef

harden

9682b60

fix docs

b9c660a

harden

57792b8

fix red CI

514ddac

Remove redundant _get_visible_tools wrapper in CodeMode

13b0fbe

Rewrite CodeMode docs with proper motivation and structure

767777f

jlowin force-pushed the code-mode branch from ba09d37 to 767777f Compare February 27, 2026 03:46

mintlify Bot deployed to staging - docs February 27, 2026 03:48 View deployment

This comment was marked as resolved.

Sign in to view

jlowin added 3 commits February 27, 2026 09:49

Fix type narrowing in collision test

bfc3afd

Simplify call_tool return type: dict | str

6279a93

mintlify Bot deployed to staging - docs February 27, 2026 16:47 View deployment

Fix example client to unwrap structured results

d1b4c50

chatgpt-codex-connector Bot reviewed Feb 27, 2026

View reviewed changes

Let server resolve tool versions instead of pinning first match

faf0893

Rewrite CodeMode docs to match current behavior

273684d

mintlify Bot deployed to staging - docs February 27, 2026 17:02 View deployment

Rename optional extra from monty to code-mode

a296d4e

mintlify Bot deployed to staging - docs February 27, 2026 17:04 View deployment

chatgpt-codex-connector Bot reviewed Feb 27, 2026

View reviewed changes

Comment thread src/fastmcp/experimental/transforms/code_mode.py

Comment thread src/fastmcp/experimental/transforms/code_mode.py

jlowin merged commit b915340 into main Feb 27, 2026
8 of 9 checks passed

jlowin deleted the code-mode branch February 27, 2026 17:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add experimental CodeMode transform#3297

Add experimental CodeMode transform#3297
jlowin merged 20 commits intomainfrom
code-mode

aaazzam commented Feb 25, 2026 •

edited by jlowin

Loading

Uh oh!

This comment was marked as resolved.

Uh oh!

marvin-context-protocol Bot commented Feb 25, 2026 •

edited

Loading

Uh oh!

This comment was marked as resolved.

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Feb 27, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

aaazzam commented Feb 25, 2026 • edited by jlowin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Design notes

Uh oh!

This comment was marked as resolved.

Uh oh!

marvin-context-protocol Bot commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Failure Analysis

Uh oh!

This comment was marked as resolved.

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aaazzam commented Feb 25, 2026 •

edited by jlowin

Loading

marvin-context-protocol Bot commented Feb 25, 2026 •

edited

Loading