Skip to content

fix(items): strip output-only created_by when replaying model output as input - #4308

Merged
seratch merged 3 commits into
openai:mainfrom
hsusul:fix/replay-strip-created-by
Aug 9, 2026
Merged

seratch merged 3 commits into
openai:mainfrom
hsusul:fix/replay-strip-created-by

Conversation

@hsusul

@hsusul hsusul commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

ModelResponse.to_input_items() replays each model output item back as a Responses API input item. Several output item types carry a server-assigned, output-only created_by field that is not part of the input-item schema:

  • apply_patch_call and shell_call (supported ApplyPatchTool / ShellTool calls)
  • function_call_output, custom_tool_call_output, computer_call_output (tool-call outputs)
  • tool_search_call / tool_search_output

The replay sanitizer in _output_item_to_input_item only stripped created_by for the two tool_search types, so every other type leaked created_by straight back to the API on replay.

created_by is not accepted by any input-item param in the OpenAI SDK schema, and the SDK already strips it at resolution time for shell/apply_patch calls (turn_resolution.py) and for tool_search (_tool_search_item_to_input_item). The direct ModelResponse.to_input_items() replay path bypassed that resolution-time stripping, leaving the field in place.

Fix: strip created_by for every replayed item in _output_item_to_input_item, generalizing the existing tool_search behavior. created_by is never read by any SDK logic (it is only ever popped), so dropping it on replay is safe.

Before:

resp = ModelResponse(output=[apply_patch_call_with_created_by], usage=Usage(), response_id=None)
resp.to_input_items()
# [{'id': 'apc_1', 'call_id': 'call_1', 'operation': {...}, 'status': 'completed',
#   'type': 'apply_patch_call', 'created_by': 'program_1'}]   # <-- leaked

After:

# [{'id': 'apc_1', 'call_id': 'call_1', 'operation': {...}, 'status': 'completed',
#   'type': 'apply_patch_call'}]   # created_by stripped

The behavior is otherwise unchanged: items are still converted via the same dict(...) / model_dump(exclude_unset=True) logic, and the AgentsException fallback for unexpected types is preserved. The only caller affected is ModelResponse.to_input_items(); the two other callers (_fingerprint_for_tracker, _anonymous_tool_search_fingerprint in oai_conversation.py) only pass tool_search items, which already routed through the created_by-stripping branch.

Test plan

  • Added test_to_input_items_strips_created_by_for_non_tool_search_items in tests/test_items_helpers.py, which builds an apply_patch_call and a function_call_output output item with created_by set and asserts the field is dropped on replay. It fails on main (leaks created_by) and passes with this change.
  • The existing test_to_input_items_for_tool_search_strips_created_by and all other to_input_items round-trip tests still pass.
  • Commands run locally:
    • uv run pytest tests/test_items_helpers.py → 36 passed
    • uv run pytest tests/test_items_helpers.py tests/memory tests/test_agent_runner.py tests/test_run_state.py tests/test_server_conversation_tracker.py tests/test_programmatic_tool_calling.py → 903 passed
    • uv run ruff format --check and uv run ruff check → clean
    • uv run python .github/scripts/check_optional_truthiness.py src/agents → clean
    • uv run mypy src/agents/items.py → Success
    • uv run pyright src/agents/items.py tests/test_items_helpers.py → 0 errors

Not run: the full test matrix and integration/coverage profiles (they require provider credentials / live services, which are unrelated to this change).

Issue number

N/A — no existing issue.

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh
  • I've confirmed all verification steps pass
  • If using Codex, I've run /review before submitting this PR

…as input

ModelResponse.to_input_items() replays each output item back as a Responses
API input item. Several output item types (apply_patch/shell calls and the
function/custom/computer tool-call outputs) carry a server-assigned created_by
field that is not part of the input-item schema. The replay sanitizer only
stripped it for tool_search items, so those other types leaked created_by back
to the API on replay.

Strip created_by for every replayed item, matching the existing tool_search
behavior and the resolution-time stripping already done for shell/apply_patch
calls. created_by is never read by SDK logic, so dropping it is safe.

@seratch seratch left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One supported Responses shape is still left unsanitized: each entry in shell_call_output.output can carry its own output-only created_by. The current helper removes only the top-level field, so ModelResponse.to_input_items() still sends input[i].output[j].created_by. The normal runner path already removes both levels in turn_resolution.py.

Please sanitize the nested shell output chunks as well without mutating the original mapping input, and add a regression test covering both top-level and nested created_by. Once the replayed request contains neither field, this should be ready for another review.

…hunks

shell_call_output carries created_by both at the item level and inside each
chunk of its output list. The replay sanitizer only removed the top-level
field, so ModelResponse.to_input_items() still leaked output[j].created_by.
Rebuild the output list with fresh chunk copies (no mutation of the caller's
original mapping), matching the two-level stripping the runner already does in
turn_resolution.
@seratch seratch added this to the 0.20.x milestone Aug 9, 2026
@seratch
seratch enabled auto-merge (squash) August 9, 2026 04:01
@seratch

seratch commented Aug 9, 2026

Copy link
Copy Markdown
Member

@hsusul Can you resolve the typecheck error?

tests/test_items_helpers.py:653: error: List item 0 has incompatible type "dict[str, Any]"; expected "ResponseOutputMessage | ResponseFileSearchToolCall | ResponseFunctionToolCall | ResponseFunctionToolCallOutputItem | ResponseFunctionWebSearch | <23 more items>"  [list-item]
Found 1 error in 1 file (checked 854 source files)

auto-merge was automatically disabled August 9, 2026 04:32

Head branch was pushed to by a user without write access

@seratch
seratch enabled auto-merge (squash) August 9, 2026 04:54
@seratch
seratch merged commit aec2dfa into openai:main Aug 9, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants