Skip to content

[Specification] Clarify message delivery order for tool-related notifications in Streamable HTTP transport #430

@LucaButBoring

Description

@LucaButBoring

Is your feature request related to a problem? Please describe.
The current specification does not a define delivery order for notifications, and explicitly states the receiver must not send any response that could be used by the server to be able to ensure message delivery. This becomes a problem when a tool using Streamable HTTP statefully wants to send notifications such as (for example) list_changed and be able to ensure that they are delivered prior to the next conversation turn.

In the 2024-11-05 SSE transport, all server messages (tool responses and notifications) were sent on the same SSE stream, meaning a server could implicitly make this guarantee by simply writing notification payloads to the stream prior to writing a tool response to the stream. However, in Streamable HTTP, this is not the case due to two key changes:

  1. Tool responses may either be sent on an open SSE stream or as the response to a client’s CallTool POST request (source). How this behaves when opening an SSE stream concurrently with a tool call is unspecified.
  2. A single client may open multiple SSE streams with the same server, using the same session (source).

The message delivery order was previously only able to be guaranteed due to always using a single response connection, but as soon as we need to write to multiple open connections for a single session, we can no longer be certain that messages are delivered to a client in the order they are written. Factors such as thread scheduling can affect the time at which data is actually written to a connection in the network stack, and network conditions can affect the time at which data is received by the client.

The TypeScript SDK seems to choose to explicitly handle this by writing tool-related notifications to the same response stream used for a tool call if possible, but this doesn’t appear to be spec-defined, it’s simply an SDK-specific choice that may not be mirrored in other language SDKs.

Describe the solution you'd like
Update the specification to explicitly discuss message delivery order for tool-related notifications across all transport methods. This could be achieved by doing any one of (from least- to most-conservative):

  1. Adding notification acknowledgments so servers can delay sending tool responses until tool-related notifications are delivered. This would be a significant change that would reverse the current no-response guidance for notifications.
  2. Requiring that tool-related notifications are always sent on the same stream as tool responses if an SSE stream is open, codifying the TypeScript SDK's current behavior and forcing other SDKs to follow suit.
  3. Noting that the delivery order of notifications is explicitly undefined, and that tool-related notifications are not guaranteed to be delivered prior to a tool response, even if some servers may ultimately offer this guarantee outside of the spec.

Describe alternatives you've considered
Continue with the current specification where delivery order is not explicitly specified and relies on the assumption of using the same connection for both tool responses and notifications. However, this assumption breaks in Streamable HTTP as it allows for multiple server-to-client message streams.

This may lead to developer confusion if they attempt to port their software from one language to another and discover that they were unknowingly relying on this behavior, or if a developer relying on this behavior in an application using the prior SSE transport upgrades to Streamable HTTP and is not using the TypeScript SDK.

Additional context
This point of ambiguity was noted in #322 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions