Python: [Bug]: Content.from_dict leaves the nested content of image generation and MCP server tool results as plain dicts #9303

Observed Behavior

After a to_dict() / from_dict() round trip, two kinds of nested content come back as plain dicts instead of Content:

• the outputs of an image_generation_tool_result. The OpenAI chat client sets it to a single Content (the generated image as a data URI).
• the items of the output list of an mcp_server_tool_result. The OpenAI and Anthropic chat clients set it to a list of Content.

This happens wherever messages are saved and loaded, for example when an AgentSession with its history is serialized with to_dict() and restored with AgentSession.from_dict(), or with Message.from_dict() directly. Code that expects Content then loses the data. For example, the Responses host reads the generated image with _image_generation_result(), which returns None for the restored result, so the image is dropped from the output item.

from_dict already rebuilds other nested content (function_call, screenshot, the inputs and outputs lists, items), but not a single outputs value or the output list.

Expected Behavior

The nested image and MCP output content comes back as Content, the same as before saving. MCP outputs that are plain JSON values stay unchanged.

Steps to Reproduce

1. Build a message with an image generation result and an MCP server tool result, as the OpenAI chat client does.
2. Save and restore it (here through an AgentSession).
3. Check the type of the nested values.

Minimal Reproduction

from agent_framework import AgentSession, Content, Message
from agent_framework_hosting_responses._parsing import _image_generation_result

message = Message("assistant", [
    Content.from_image_generation_tool_result(
        image_id="ig_1", outputs=Content.from_uri("data:image/png;base64,iVBORw0KGgo=")
    ),
    Content.from_mcp_server_tool_result("mcp_1", output=[Content.from_text("42")]),
])

session = AgentSession()
session.state["history"] = {"messages": [message]}
restored = AgentSession.from_dict(session.to_dict()).state["history"]["messages"][0]

print(type(restored.contents[0].outputs).__name__)    # dict (expected Content)
print(type(restored.contents[1].output[0]).__name__)  # dict (expected Content)
print(_image_generation_result(message.contents[0].outputs))   # iVBORw0KGgo=
print(_image_generation_result(restored.contents[0].outputs))  # None

The cause is in Content.from_dict in agent_framework/_types.py: it rebuilds outputs only when it is a list, and it does not look at output at all.

Error Messages and Stack Traces

No error. The restored values are plain dicts, so the image or tool output is silently lost where Content is expected.

Package Versions

agent-framework-core: 1.21.0 (main at a75e803), agent-framework-hosting-responses: 1.0.0a261008

Python Version

Python 3.12

Additional Context

I'd like to fix this. In Content.from_dict, also rebuild a single outputs mapping and the items of an output list, but only mappings whose type is a known content type, so plain MCP outputs stay as they are. Plus regression tests.

AI assistance: I used an AI coding assistant for the analysis and to draft this report. I reproduced and verified the behavior myself.