Skip to content

fix(caching): partition handler-result caches by MCP session - #5576

Closed
BlueX888 wants to merge 1 commit into
PrefectHQ:mainfrom
BlueX888:fix/prep-caching-call-tool-session-state-leak
Closed

BlueX888 wants to merge 1 commit into
PrefectHQ:mainfrom
BlueX888:fix/prep-caching-call-tool-session-state-leak

Conversation

@BlueX888

@BlueX888 BlueX888 commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Description

ResponseCachingMiddleware partitioned its cache entries by the caller's access token, but not by MCP session. Session-scoped state is per-session by contract — Context.set_state / get_state key it by session id — so with the middleware installed, a second session's tools/call was served the first session's cached answer even after it had stored its own value, and likewise for resources/read / prompts/get handlers that read state. The token partition added for #4037 stopped one user's filtered view reaching another user; this was the same defect one level down, inside a single token partition.

The tools/call, resources/read, and prompts/get cache keys now also mix in the session's established id, resolved by a new Context._established_session_id that follows session_id's resolution order without its generate-and-cache tail — so transports with no stable session identity keep sharing a session-less partition instead of losing the cache entirely. List methods keep their per-token key: list handlers never observe session state.

server = FastMCP()
server.add_middleware(ResponseCachingMiddleware())

@server.tool
async def remember(key: str, value: str, ctx: Context) -> str:
    await ctx.set_state(key, value)
    return "stored"

@server.tool
async def recall(key: str, ctx: Context) -> str:
    return await ctx.get_state(key) or "<unset>"

# session A: remember("k", "a") then recall -> "a" (cached under A's session partition)
# session B: remember("k", "b") then recall -> "b" (B's own state, never A's cached answer)

Testing, on main @ 0ebabc0: the new TestSessionAwareCaching regression tests fail without the change (assert 'session-a-secret' == 'session-b-secret') and pass with it; uv run pytest tests/server/middleware/test_caching.py → 80 passed; uv run pytest tests/server/ tests/tasks/ → 4160 passed with 8 pre-existing proxy-network failures that fail identically on unchanged main; uv run prek run --all-files → all enforced checks pass.

Closes #5574

Generated with AI assistance (GLM agent); found through local code inspection and reproduction.

Contribution type

  • Bug fix (simple, well-scoped fix for a clearly broken behavior)
  • Documentation improvement
  • Enhancement (maintainers typically implement enhancements — see CONTRIBUTING.md)

Checklist

  • This PR addresses an existing issue (or fixes a self-evident bug)
  • I have read CONTRIBUTING.md
  • I have added tests that cover my changes
  • I have run uv run prek run --all-files and all checks pass
  • I have self-reviewed my changes
  • If I used an LLM, it followed the repo's contributing conventions (not generic output)

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

This pull request is awaiting maintainer assignment. Linked PRs stay open while maintainers triage the issue; this check remains failing until assignment.

Per CONTRIBUTING.md, an external PR must reference an issue that's assigned to its author. To get there:

  1. A maintainer will review the issue and decide whether to assign you. Please don't comment just to request assignment.

Once you're assigned and the link is present, this check is re-run automatically. Previously gate-closed PRs also reopen — no further action needed.

Maintainers: reopen this PR or remove the missing-issue-link label to bypass this check.

Generated with AI assistance (GLM agent).
@BlueX888
BlueX888 force-pushed the fix/prep-caching-call-tool-session-state-leak branch from cd56de8 to bdb0257 Compare October 8, 2026 10:15
@zzstoatzz

Copy link
Copy Markdown
Collaborator

Closing with #5574: sessions without a token sharing a cache partition is the intended behavior.

@zzstoatzz zzstoatzz closed this Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ResponseCachingMiddleware serves one session's cached tools/call result to another session

2 participants