There's a level of ambiguity here, but I think that rough mental model should fit most conversations. But for the purpose of explaining the overlap - ImageContent is part of the conversation and ends up with the LLM. EmbeddedResource containing an Image with an audience of user would be rendered to the User, but with no hard expectation of it ending up in the LLMs context. This gives a flexible way to manage a range of Content types and indicate certain content for User Display/Host Processing but not necessarily LLM context.

To me, it seems like audience is the key distinguisher. EmbeddedResource vs ImageContent, AudioContent, etc doesn't seem to be a meaningful distinction.

Do you think things would be more clear if we had a unified content item type with appropriate metadata? Something like:

export interface ContentItem {
  mimeType: string

  // Plain text or base64-encoded binary, depending on `mimeType`.
  data: string

  // All annotations are optional.
  annotations?: {
    // "base64" if `data` is base64-encoded binary; otherwise null.
    // If omitted, client may assume `data` is plain text.
    encoding?: "base64" | null;

    // Indicates whether the item is for user or model or both.
    // If for model, the item should be added to the context.
    // If omitted, client may assume the item is for both user and model.
    audience?: Role[];

    // Hint for the client about how the item should be rendered.
    // If omitted, client may assume "inline".
    //
    // Possibly support other values here, e.g. "ui" for dynamic UI widgets?
    disposition?: "inline" | "attachment";

    // Description of the item, targeted at either the user or the model (or both).
    //
    // Possibly require when `disposition: "attachment"` for use as a label?
    description?: string;

    // URL for the item.  May be an ephemeral URL.
    url?: string;

    // Hint for the client that `url` is ephemeral.
    // Only a hint -- false does not guarantee that the item will be accessible
    // via the URL in the future, and true does not guarantee that the item
    // will become inaccessible.
    ephemeral?: boolean;

    // A reference to a schema that `data` conforms to.
    // Neither the format of the reference nor the format of the schema is
    // specified.  If the receiver recognizes the format of the reference and
    // the format of the schema, then the schema may be used for validation;
    // however, the primary purpose of `schemaRef` is to identify the shape of
    // `data` beyond what `mimeType` is able to convey.
    //
    // Example: a canonical URL of a JSON schema when `mimeType: "application/json"`.
    //
    // Possibly provide more guidance here for popular MIME types?
    schemaRef?: string;
  }
}

The above ContentItem type would replace EmbeddedResource and all *Content types.

Personal opinion - yes, something like this would be an improvement, and also mean that future modalities such as Video would slot-in without any real change. This scheme would also be supportable for a transitional period where API compatibility could be maintained.

There's plenty to discuss over the specifics of what that may look like, as well as some stronger guidance over the transmission of "files" to be done at the same time I would say.

More personal opinion - this isn't something that needs to be rushed: the status quo isn't restrictive - the missing thing is understanding of the existing options. As HTTP Streaming becomes more established for example, it would be useful to have some time to consider what roots mean in that context and whether the file:// uri restriction should be removed or have MCP Servers be able to request "Root Resources" from the MCP Client. This is the kind of thing where I think a Community Working Group could add a lot of value as real-world experience is gained, and we can consider the end-to-end effects of proposals from Spec -> SDK -> Hosting -> Client -> Host -> Human. This is also an area where a couple of light touches like #223 can help.

The one thing I would argue strongly to maintain is a simple default TextContent type that is known to be text/plain and used as the general default.

Keeping it with this thread (I'll need to search again later), there was also an important conversation about just using the uri as a pointer. So, for example, rather than have the text|blob in the EmbeddedResource you would instead just send the uri with the expectation the Client would fetch it. This saves transmitting potentially large amounts of data as part of a CallToolResult (so for example you might generate an image, and instead of using EmbeddedResource you might return AttachedResource with a uri of https://image-server/whatever.png. I'll see if I can find the original thread.

Keeping it with this thread (I'll need to search again later), there was also an important conversation about just using the uri as a pointer. So, for example, rather than have the text|blob in the EmbeddedResource you would instead just send the uri with the expectation the Client would fetch it. This saves transmitting potentially large amounts of data as part of a CallToolResult (so for example you might generate an image, and instead of using EmbeddedResource you might return AttachedResource with a uri of https://image-server/whatever.png. I'll see if I can find the original thread.

Ah, that's an interesting use case! That could be very useful in agentic workflows where there may be several layers of message forwarding.

Then I think it would make sense to move url to the top-level alongside data. (And add a stipulation that at least one of the two must be present, and if both are present, they should represent the same content.)

Adding some notes here:

• Instead of the ephemeral hint, support a urlScope hint with possible values like:
  • "local" - URL is for a local resource, e.g. a file (no liveness guarantee).
  • "remote" - URL is for a remote resource, i.e. requires network access (no liveness guarantee).
  • "server" - URL is for a server resource, i.e. can be fetched with resources/read (no liveness guarantee).
    • Possibly add a "server/listed" value that does guarantee liveness as long as resources/list lists the URL?
  • "client" - URL is for a resource that is managed by the client.
    • The idea here is to enable the model to refer to resources that are managed entirely by the client. Possibly require a special URL scheme like mcp-client-storage://{uuid}?
• A better name for schemaRef might be conformsTo in order to convey the loose semantics of that property. Another thought is that it could be a descriptive object instead of a string, e.g. conformsTo: { jsonSchema: "url" } }. (I don't think the value should be the schema object itself, but I suppose forbidding that would be untenable.)

I think these are all good ideas; and with A2A etc. starting to form/be implemented I think it will become easier to make good judgements over what fits best. @tadasant -- I think this would make quite a good CWG topic - shall I set one up?