Skip to content

Commit 03626b2

Browse files
committed
gateway: an item a batch's tool results were interrupted by goes after them, so strict Responses upstreams stop refusing the relayed request
What: the normalization a relayed Responses request goes through before it leaves (gateway.go, attempt) gains joinToolRuns beside orphanedToolOutputs and pairToolItems: an item that sits inside a run of tool calls and results - between a call and the result that answers it - is moved to the end of that run. A run is open while one of its calls has been seen and the result answering it is still ahead, so an item between two whole exchanges stays where it is. Codex writes such an item itself: when a tool returned an image it resized, its <image_resize_notice> goes between one batch's two results (the report from the Codex app on Windows, one assistant step with view_image and exec_command). Before, the run reached the upstream with the item inside it and a strict one refused the whole request with 400: Volcengine's Agent Plan "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'", OpenCode Zen/Go "No tool output found for tool call <id>" - both naming a result that is in the request. After, the results follow the calls, the item goes after them, and both answer 200. This is the rule the chat path has had since yetone#1275 (pairToolMessages closes a run with its results, interrupted ones included, before another message) and the same family as yetone#1341 (pairToolItems, a call nothing answers), and it is applied before the request is relayed or translated, so every destination whose own form is strict gets it in the same change. The item's own bytes, its role and its order among the other non-tool items are as they were. A request with no tool items, one whose exchanges are each whole, and a body that is not a Responses request come back byte for byte (TestJoinToolRunsLeavesWholeExchangesAlone). One expectation changed on purpose: TestGrokGetsAgentMessageAsUser's turn holds a subagent's reply between the send_message call and its output, and Grok is now given that reply after the run. Its conversion is unchanged (same user message, same "From <author> to <recipient>" header, same text and images) and only its place moved, the way every other item Grok is handed keeps the place magpie gave it; a review that knows Grok's backend needs the reply in place can narrow the rule to messages other than agent_message. Run: - go test -tags nogui ./internal/gateway/: 2 failures, both red on origin/main 6c1c0a7 as well and neither from this change - TestClaudeAsRootEnv (a Linux-only expectation with no Windows guard) and TestSweepBridgeProjectsTakesOnlyTheBridgesFolders (needs a symlink privilege this Windows box doesn't hold). Everything else passes, TestQueuedStreamKeptAliveOnceSentToItsVendor among them (that one failed once on main under load; it is the queued-stream timing in the standards). - the same run under a temporary HOME with GOPATH/GOMODCACHE/GOCACHE pinned from go env: the same two failures. - gofmt -l on the four changed files: nothing. go vet ./..., GOOS=linux go vet -tags nogui ./..., GOOS=windows go vet ./...: clean. go build, GOOS=windows go build, GOOS=linux go build -tags nogui: ok. - without the change: with the one call-site line taken out, TestResponsesRelayKeepsAToolRunTogether prints `status 400: ... "Fake: An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'."`; the line back, it passes. - real upstreams, the reporter's shape (two parallel calls, an image result, Codex's <image_resize_notice>, the second result), the same build with the line and without it, streamed the way Codex sends it: Volcengine Agent Plan / deepseek-v4.1-flash - run whole 200/200; the notice between the results 400 "insufficient tool messages following tool_calls message" then 200; any other note there 400 then 200. OpenCode Zen/Go / deepseek-v4.1-flash - run whole 200/200; a note between the results 400 "No tool output found for tool call call_b" then 200; the notice between them 200/200, their gateway recognizing that one item. - the reporter's own rebuilt turn (1399 items, from their rollout) to Volcengine: refused "insufficient tool messages following tool_calls message" without the change, past that check with it (what is left is the input's length, 1.7M tokens, over the model's limit). Not tried: a real Grok subscription - the reply's new place is unverified against xAI's backend, though its conversion is unchanged and its plugin (community plugins packages/grok) converts in place the same way; a real ChatGPT (native Codex account) destination, which takes standalone tool outputs itself and is left as it came; the Windows GUI. The OpenCode Zen/Go 400 of the original report is not reproduced by the reporter's own bytes: with both results and the notice between them that gateway answers 200 before and after this change, so the report's refusal may be the follow-up body assembled before the batch's last output was in it (openai/codex#44604). Refs: yetone#1341, yetone#1275. Reported as anomalyco/opencode#54698 and openai/codex#44604.
1 parent d0c587a commit 03626b2

4 files changed

Lines changed: 266 additions & 11 deletions

File tree

‎internal/gateway/gateway.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2908,6 +2908,7 @@ func (s *Server) attempt(w http.ResponseWriter, r *http.Request, from provider.P
29082908
// Normalize for the actual destination, separately on each fallback.
29092909
// Native ChatGPT accounts accept standalone tool outputs themselves.
29102910
if from == provider.Responses && (p.Account == nil || p.Account.Agent != "codex") {
2911+
body = joinToolRuns(body)
29112912
body = orphanedToolOutputs(body)
29122913
body = pairToolItems(body)
29132914
}

‎internal/gateway/grok_agent_message_test.go‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,21 +22,24 @@ const agentMessageTurn = `"input":[
2222
{"type":"function_call_output","call_id":"c1","output":"sent"}]`
2323

2424
// checkAgentMessageSent fails unless Grok was given the subagent's reply
25-
// as the user's message, in its place, its sender and recipient first.
25+
// as the user's message, after the run it broke, its sender and recipient
26+
// first. The reply sat between the call and its output on the way in;
27+
// joinToolRuns puts it after the output, so the exchange Grok reads is
28+
// whole. Its text, its role, and its conversion are as they were.
2629
func checkAgentMessageSent(t *testing.T, q map[string]any) {
2730
t.Helper()
2831
in, _ := q["input"].([]any)
2932
if len(in) != 4 {
3033
t.Fatalf("Grok was given %v", q["input"])
3134
}
32-
m, _ := in[2].(map[string]any)
35+
if o, _ := in[2].(map[string]any); o["type"] != "function_call_output" || o["call_id"] != "c1" {
36+
t.Fatalf("its call's output went as %v", in[2])
37+
}
38+
m, _ := in[3].(map[string]any)
3339
got, _ := json.Marshal(m)
3440
if want := `{"content":[{"text":"From /root/research to /root\n\n","type":"input_text"},{"text":"Message Type: FINAL_ANSWER\nPayload:\nfound the cause","type":"input_text"}],"role":"user","type":"message"}`; string(got) != want {
3541
t.Fatalf("the agent_message went as\n%s\nwant\n%s", got, want)
3642
}
37-
if o, _ := in[3].(map[string]any); o["type"] != "function_call_output" || o["call_id"] != "c1" {
38-
t.Fatalf("its call's output went as %v", in[3])
39-
}
4043
}
4144

4245
// A Codex subagent's plain agent_message reaches a Grok subscription as the
Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
package gateway
2+
3+
import (
4+
"bytes"
5+
"encoding/json"
6+
"testing"
7+
8+
"github.com/yetone/magpie/internal/provider"
9+
)
10+
11+
// notice is the item Codex writes between one batch's tool results when the
12+
// image a tool returned was resized for the model.
13+
const resizeNotice = `<image_resize_notice>
14+
Image 1 of 1 in the preceding tool output was resized from 1024x9000 to 233x2048 pixels.
15+
</image_resize_notice>`
16+
17+
// orderOf is a Responses request's input items as "type:call_id" strings.
18+
func orderOf(t *testing.T, body []byte) []string {
19+
t.Helper()
20+
var order []string
21+
for _, it := range inputOf(t, body) {
22+
typ, _ := it["type"].(string)
23+
id, _ := it["call_id"].(string)
24+
order = append(order, typ+":"+id)
25+
}
26+
return order
27+
}
28+
29+
// A batch of two calls whose results have a message between them: the
30+
// message goes after both results, where the run a strict upstream checks
31+
// is whole. Fixtures are the reporter's own bytes — two parallel calls
32+
// (view_image and exec_command), an output holding an image, Codex's resize
33+
// notice, and the second output.
34+
func TestJoinToolRunsMovesAnItemOutOfARun(t *testing.T) {
35+
body := []byte(`{"model":"m","input":[` +
36+
`{"type":"message","role":"user","content":[{"type":"input_text","text":"look"}]},` +
37+
`{"type":"function_call","call_id":"call_00_hC0DonBjeKHawzABcyLL1642","name":"view_image","arguments":"{\"path\":\"wide.png\"}"},` +
38+
`{"type":"function_call","call_id":"call_01_mNySew0rARWuQYQXPeIY2319","name":"exec_command","arguments":"{\"cmd\":\"python -c ...\"}"},` +
39+
`{"type":"function_call_output","call_id":"call_00_hC0DonBjeKHawzABcyLL1642","output":[{"type":"input_image","image_url":"data:image/png;base64,AQID"}]},` +
40+
`{"type":"message","role":"developer","content":[{"type":"input_text","text":` + jsonStr(resizeNotice) + `}]},` +
41+
`{"type":"function_call_output","call_id":"call_01_mNySew0rARWuQYQXPeIY2319","output":"PIL 11.3.0\r\n"}]}`)
42+
original := bytes.Clone(body)
43+
got := joinToolRuns(body)
44+
if !bytes.Equal(body, original) {
45+
t.Fatal("mutated the caller's request buffer")
46+
}
47+
want := []string{
48+
"message:",
49+
"function_call:call_00_hC0DonBjeKHawzABcyLL1642",
50+
"function_call:call_01_mNySew0rARWuQYQXPeIY2319",
51+
"function_call_output:call_00_hC0DonBjeKHawzABcyLL1642",
52+
"function_call_output:call_01_mNySew0rARWuQYQXPeIY2319",
53+
"message:",
54+
}
55+
order := orderOf(t, got)
56+
if len(order) != len(want) {
57+
t.Fatalf("items %v, want %v", order, want)
58+
}
59+
for i := range want {
60+
if order[i] != want[i] {
61+
t.Fatalf("items %v, want %v", order, want)
62+
}
63+
}
64+
// the notice keeps its text, and the results keep theirs
65+
if !bytes.Contains(got, []byte("image_resize_notice")) ||
66+
!bytes.Contains(got, []byte("PIL 11.3.0")) ||
67+
!bytes.Contains(got, []byte("data:image/png;base64,AQID")) {
68+
t.Fatalf("lost an item's content: %s", got)
69+
}
70+
// the same request again is left as it is
71+
if again := joinToolRuns(got); !bytes.Equal(again, got) {
72+
t.Fatalf("not idempotent:\n%s\n%s", got, again)
73+
}
74+
}
75+
76+
// Only an item that breaks a run moves: one between two whole exchanges, a
77+
// request with no tool items, and anything that isn't a Responses body go
78+
// byte for byte.
79+
func TestJoinToolRunsLeavesWholeExchangesAlone(t *testing.T) {
80+
for _, keep := range []string{
81+
// each exchange is whole: the call before the notice is answered,
82+
// and so is the one after it
83+
`{"model":"m","input":[` +
84+
`{"type":"function_call","call_id":"call_a","name":"echo","arguments":"{}"},` +
85+
`{"type":"function_call_output","call_id":"call_a","output":"a"},` +
86+
`{"type":"message","role":"developer","content":[{"type":"input_text","text":"a note"}]},` +
87+
`{"type":"function_call","call_id":"call_b","name":"echo","arguments":"{}"},` +
88+
`{"type":"function_call_output","call_id":"call_b","output":"b"}]}`,
89+
// a call nothing answers is not a run to keep together
90+
`{"model":"m","input":[` +
91+
`{"type":"function_call","call_id":"call_a","name":"echo","arguments":"{}"},` +
92+
`{"type":"message","role":"user","content":[{"type":"input_text","text":"never mind"}]}]}`,
93+
// parallel results already contiguous, and out of the calls' order
94+
`{"model":"m","input":[` +
95+
`{"type":"function_call","call_id":"call_a","name":"echo","arguments":"{}"},` +
96+
`{"type":"function_call","call_id":"call_b","name":"echo","arguments":"{}"},` +
97+
`{"type":"function_call_output","call_id":"call_b","output":"b"},` +
98+
`{"type":"function_call_output","call_id":"call_a","output":"a"}]}`,
99+
`{"model":"m","input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"hi"}]}]}`,
100+
`not json`, `{}`, `{"input":"hello"}`,
101+
} {
102+
if got := joinToolRuns([]byte(keep)); string(got) != keep {
103+
t.Fatalf("rewritten:\n%s\nto\n%s", keep, got)
104+
}
105+
}
106+
}
107+
108+
// The relay keeps the run whole for the upstream: a request whose batch has
109+
// an item between its results comes out with the results together, and an
110+
// upstream that checks the exchange answers it.
111+
func TestResponsesRelayKeepsAToolRunTogether(t *testing.T) {
112+
f := &fake{t: t, reply: sse(`data: {"type":"response.completed","response":{"id":"r1","status":"completed","output":[]}}`)}
113+
f.refuse = func(body []byte) (int, string) {
114+
// The upstream's own check, on the chat form it builds: every result
115+
// of a run must follow the calls without an item between them.
116+
open := 0
117+
var sent struct {
118+
Input []struct {
119+
Type string `json:"type"`
120+
CallID string `json:"call_id"`
121+
} `json:"input"`
122+
}
123+
if err := json.Unmarshal(body, &sent); err != nil {
124+
t.Fatal(err)
125+
}
126+
for _, it := range sent.Input {
127+
switch it.Type {
128+
case "function_call":
129+
open++
130+
case "function_call_output":
131+
open--
132+
default:
133+
if open > 0 {
134+
return 400, `{"error":{"message":"An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'."}}`
135+
}
136+
}
137+
}
138+
return 0, ""
139+
}
140+
setup(t, provider.Responses, f)
141+
body := `{"model":"fake/m1","stream":true,"input":[` +
142+
`{"type":"message","role":"user","content":[{"type":"input_text","text":"look"}]},` +
143+
`{"type":"function_call","call_id":"call_a","name":"view_image","arguments":"{}"},` +
144+
`{"type":"function_call","call_id":"call_b","name":"exec_command","arguments":"{}"},` +
145+
`{"type":"function_call_output","call_id":"call_a","output":"looked"},` +
146+
`{"type":"message","role":"developer","content":[{"type":"input_text","text":` + jsonStr(resizeNotice) + `}]},` +
147+
`{"type":"function_call_output","call_id":"call_b","output":"ran"}]}`
148+
if code, out := post(t, "/v1/responses", body); code != 200 {
149+
t.Fatalf("status %d: %s", code, out)
150+
}
151+
want := []string{"message:", "function_call:call_a", "function_call:call_b",
152+
"function_call_output:call_a", "function_call_output:call_b", "message:"}
153+
order := orderOf(t, f.got)
154+
if len(order) != len(want) {
155+
t.Fatalf("sent %v, want %v", order, want)
156+
}
157+
for i := range want {
158+
if order[i] != want[i] {
159+
t.Fatalf("sent %v, want %v", order, want)
160+
}
161+
}
162+
}

‎internal/gateway/responses.go‎

Lines changed: 95 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -481,6 +481,100 @@ func parseResponses(body []byte) (*Request, error) {
481481
return r, nil
482482
}
483483

484+
// toolCallItem is a Responses input item that makes a tool call, and
485+
// toolResultItem one that answers a call.
486+
func toolCallItem(t string) bool {
487+
switch t {
488+
case "function_call", "custom_tool_call", "tool_search_call":
489+
return true
490+
}
491+
return false
492+
}
493+
494+
func toolResultItem(t string) bool {
495+
switch t {
496+
case "function_call_output", "custom_tool_call_output", "tool_search_output":
497+
return true
498+
}
499+
return false
500+
}
501+
502+
// joinToolRuns moves an item that sits inside a run of tool calls and
503+
// results — between a call and the result that answers it — to the end of
504+
// that run. An upstream that checks the exchange, on the chat form it turns
505+
// a relayed request into or on the Responses input itself, reads the run as
506+
// ended there and refuses it: Volcengine's coding plan answers 400 "An
507+
// assistant message with 'tool_calls' must be followed by tool messages
508+
// responding to each 'tool_call_id'", the same refusal pairToolItems mends
509+
// a missing result for (#1341), and OpenCode Zen/Go answers "No tool output
510+
// found for tool call <id>". An agent writes an item between one batch's
511+
// results itself: Codex writes its <image_resize_notice> there when a tool
512+
// returned an image it resized. An item between two exchanges that are each
513+
// whole is left where it is, and a request with nothing inside a run comes
514+
// back byte for byte.
515+
func joinToolRuns(body []byte) []byte {
516+
if !bytes.Contains(body, []byte(`_call"`)) {
517+
return body
518+
}
519+
var q map[string]json.RawMessage
520+
if decodeJSON(body, &q) != nil {
521+
return body
522+
}
523+
var input []json.RawMessage
524+
if decodeJSON(q["input"], &input) != nil {
525+
return body
526+
}
527+
type item struct {
528+
Type string `json:"type"`
529+
CallID string `json:"call_id"`
530+
}
531+
items := make([]item, len(input))
532+
answered := map[string]bool{} // call IDs a result in the request answers
533+
for i, raw := range input {
534+
if decodeJSON(raw, &items[i]) != nil {
535+
items[i] = item{}
536+
}
537+
if toolResultItem(items[i].Type) && items[i].CallID != "" {
538+
answered[items[i].CallID] = true
539+
}
540+
}
541+
out := make([]json.RawMessage, 0, len(input))
542+
var held []json.RawMessage
543+
moved := false
544+
// open: calls of the run in progress whose result is still ahead. A run
545+
// is open while one of its calls has been seen and its result has not.
546+
open := map[string]bool{}
547+
for i, raw := range input {
548+
t, id := items[i].Type, items[i].CallID
549+
switch {
550+
case toolCallItem(t):
551+
if id != "" && answered[id] {
552+
open[id] = true
553+
}
554+
case toolResultItem(t) && id != "":
555+
delete(open, id)
556+
case len(open) > 0:
557+
held, moved = append(held, raw), true
558+
continue
559+
}
560+
out = append(out, raw)
561+
if len(open) == 0 && len(held) > 0 {
562+
out = append(out, held...)
563+
held = nil
564+
}
565+
}
566+
out = append(out, held...)
567+
if !moved {
568+
return body
569+
}
570+
q["input"], _ = marshalPlain(out)
571+
encoded, err := marshalPlain(q)
572+
if err != nil {
573+
return body
574+
}
575+
return encoded
576+
}
577+
484578
// orphanedToolOutputs turns a tool result with no call ID into a user message.
485579
// Codex uses standalone outputs for cross-thread notifications. Backends
486580
// requiring paired outputs still need the delivered text, without a fake call ID.
@@ -602,12 +696,7 @@ func pairToolItems(body []byte) []byte {
602696
}
603697
}
604698
exchange := func(t string) bool {
605-
switch t {
606-
case "function_call", "custom_tool_call", "tool_search_call",
607-
"function_call_output", "custom_tool_call_output", "tool_search_output":
608-
return true
609-
}
610-
return false
699+
return toolCallItem(t) || toolResultItem(t)
611700
}
612701
out := make([]json.RawMessage, 0, len(input)+1)
613702
var synthetic []json.RawMessage // results owed by the run in progress

0 commit comments

Comments
 (0)