Skip to content

Commit bf84d17

Browse files
olddonkeyclaude
andcommitted
fix(responses): close the remaining private-shape leaks on the routed boundary
The namespace boundary lowered complete groups but still let several Codex-private shapes reach a strict gateway, each reproducing the pre-inference rejection the boundary exists to prevent. No `type: "namespace"` value survives now. A group the layer cannot express — empty, nested, or with an unusable child name — is dropped along with the children it cannot represent. Relaying the private shape costs the whole request rather than one tool, so "preserve rather than lose a tool" was losing strictly more. Replayed call items are lowered whether or not this turn declares the group they name. The routed compaction turn strips the entire tool surface before the boundary runs, so every compaction after a namespaced tool call shipped the private `namespace` key this layer's own restoration had stamped on the item. Only tool_choice resolves a bare name through the catalog: a history item records which tool actually ran, so re-pointing it at a same-named namespace child would rewrite that record on a coincidence rather than translate it. Codex-private tool fields now come from one table instead of one bespoke pass each, and it gains `defer_loading` — `activateDeferredTool` clears that only for tools a `tool_search_output` already loaded, so the first turn of a deferred catalog carried it to the wire — and the `web_search_preview` variant. A bare declaration and a `functions` child of the same name are one logical tool: `buildTools` flattens the reserved group without a namespace, the parser tolerates the duplicate, and `promoteClientLoadedTools` produces it. That shape raised a wire-name collision that escaped every catch up to the Bun handler, so an ordinary catalog became an unstructured 500 with no request log — while the rotation-rebuild path answered 400 for the identical throw. It is now deduped, and a genuine collision is a typed error the passthrough maps to 400. Co-Authored-By: Claude Fable 5 <[email protected]>
1 parent e4508d0 commit bf84d17

6 files changed

Lines changed: 335 additions & 114 deletions

File tree

‎src/adapters/openai-responses.ts‎

Lines changed: 30 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -122,28 +122,42 @@ function stripInvalidItemIds(body: unknown): unknown {
122122
}
123123

124124
/**
125-
* Codex attaches ChatGPT's private `external_web_access` policy bit to the public
126-
* `web_search` tool. Third-party Responses APIs enable browsing by the presence of the tool and
127-
* commonly reject the extra argument (xAI returns `Argument not supported:
128-
* external_web_access`). Keep the hosted tool and every public option, but remove only that
129-
* canonical-only hint before a routed request reaches a non-OpenAI gateway.
125+
* Codex-private tool fields that only the ChatGPT backend understands.
126+
*
127+
* A third-party Responses gateway validates its schema and rejects the whole request before
128+
* inference — xAI answers `Argument not supported: external_web_access` — so these are removed at
129+
* the noncanonical boundary while the tool and every public option stay.
130+
*
131+
* Keep this a table. Each private bit Codex attaches has so far arrived as its own bespoke strip
132+
* with its own traversal, and the traversals disagreed about which containers they covered; a new
133+
* one should be a row here instead. `toolTypes` omitted means the field is private on any tool.
130134
*/
131-
function stripCanonicalWebSearchAccessHint(body: unknown): unknown {
135+
const CANONICAL_ONLY_TOOL_FIELDS: readonly { field: string; toolTypes?: ReadonlySet<string> }[] = [
136+
// ChatGPT's browsing policy bit. The public hosted tool is enabled by its presence alone.
137+
{ field: "external_web_access", toolTypes: new Set(["web_search", "web_search_preview"]) },
138+
// Deferred-discovery marker. `activateDeferredTool` clears it only for tools a `tool_search_output`
139+
// already loaded, so a still-deferred declaration — including one promoted out of a namespace
140+
// group — otherwise reaches the wire carrying it.
141+
{ field: "defer_loading" },
142+
];
143+
144+
function stripCanonicalOnlyToolFields(body: unknown): unknown {
132145
if (!isPlainObject(body)) return body;
133146

134147
const rewriteTools = (tools: unknown[]): unknown[] => {
135148
let changed = false;
136149
const rewritten = tools.map(tool => {
137-
if (
138-
!isPlainObject(tool)
139-
|| tool.type !== "web_search"
140-
|| !Object.hasOwn(tool, "external_web_access")
141-
) {
142-
return tool;
150+
if (!isPlainObject(tool)) return tool;
151+
let next = tool;
152+
for (const { field, toolTypes } of CANONICAL_ONLY_TOOL_FIELDS) {
153+
if (!Object.hasOwn(next, field)) continue;
154+
if (toolTypes && (typeof next.type !== "string" || !toolTypes.has(next.type))) continue;
155+
const { [field]: _private, ...rest } = next;
156+
next = rest;
143157
}
158+
if (next === tool) return tool;
144159
changed = true;
145-
const { external_web_access: _externalWebAccess, ...rest } = tool;
146-
return rest;
160+
return next;
147161
});
148162
return changed ? rewritten : tools;
149163
};
@@ -1627,9 +1641,8 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig):
16271641
const rewritten = rewriteRoutedNamespaceToolsForUpstream(outBody);
16281642
outBody = rewritten.body;
16291643
convertedRoutedNamespaceToolAliases = rewritten.aliases;
1630-
}
1631-
if (!isCanonicalOpenAiForwardProvider(provider)) {
1632-
outBody = stripCanonicalWebSearchAccessHint(outBody);
1644+
// Last, so promoted namespace children are also cleared of Codex-private fields.
1645+
outBody = stripCanonicalOnlyToolFields(outBody);
16331646
}
16341647
const sanitizedBody = normalizeToolSchemas(stripSparkCompatibility(stripUnsupportedReasoningParams(stripItemIdsWhenUnstored(stripInvalidItemIds(stripUnsupportedHostedTools(sanitizeReasoningInputContent(scrubOcxCompactionItems(outBody), { preserveRawReasoningContent: provider.preserveResponsesReasoningContent === true })))))));
16351648
const finalBody = stripDisabledReasoningSummaries(

‎src/responses/namespace-tool-compat.ts‎

Lines changed: 138 additions & 77 deletions
Original file line numberDiff line numberDiff line change
@@ -18,32 +18,64 @@ function namespaceIdentity(namespace: string, name: string): string {
1818
return `${namespace}\u0000${name}`;
1919
}
2020

21-
function namespaceToolIdentity(tool: unknown): RoutedNamespaceToolIdentity | undefined {
21+
/**
22+
* A name that can become a wire tool name. Control characters are rejected because the identity
23+
* key below joins namespace and name with NUL: a name carrying one could otherwise forge another
24+
* tool's identity and silently take over its wire name.
25+
*/
26+
function isRepresentableName(name: unknown): name is string {
27+
if (typeof name !== "string" || name.length === 0) return false;
28+
for (let index = 0; index < name.length; index += 1) {
29+
const code = name.charCodeAt(index);
30+
// C0 controls and DEL, written as code points so this source never carries one itself.
31+
if (code <= 0x1f || code === 0x7f) return false;
32+
}
33+
return true;
34+
}
35+
36+
type NamespaceGroup = {
37+
namespace: string;
38+
/** Children that can be lowered to a flat declaration; unrepresentable ones are omitted. */
39+
children: Record<string, unknown>[];
40+
};
41+
42+
/**
43+
* Read a private namespace group, or return undefined when the value is not one.
44+
*
45+
* Children that cannot be expressed as a flat declaration — a nested group, a missing name, a
46+
* control character in the name — are dropped, and a group left with no children is dropped whole
47+
* by the rewrite. Preserving the private `namespace` shape instead would lose every tool in the
48+
* request rather than one: the strict gateways this layer exists for reject that tool type before
49+
* inference, which is the failure the layer was written to prevent.
50+
*/
51+
function parseNamespaceGroup(tool: unknown): NamespaceGroup | undefined {
2252
if (
2353
!isPlainObject(tool)
2454
|| tool.type !== "namespace"
25-
|| typeof tool.name !== "string"
26-
|| tool.name.length === 0
55+
|| !isRepresentableName(tool.name)
2756
|| !Array.isArray(tool.tools)
28-
|| tool.tools.length === 0
2957
) return undefined;
30-
return { namespace: tool.name, name: "" };
31-
}
32-
33-
function namespaceChildren(tool: unknown): Record<string, unknown>[] | undefined {
34-
const identity = namespaceToolIdentity(tool);
35-
if (!identity || !isPlainObject(tool) || !Array.isArray(tool.tools)) return undefined;
3658
const children: Record<string, unknown>[] = [];
3759
for (const child of tool.tools) {
38-
if (
39-
!isPlainObject(child)
40-
|| child.type === "namespace"
41-
|| typeof child.name !== "string"
42-
|| child.name.length === 0
43-
) return undefined;
60+
if (!isPlainObject(child) || child.type === "namespace" || !isRepresentableName(child.name)) continue;
4461
children.push(child);
4562
}
46-
return children;
63+
return { namespace: tool.name, children };
64+
}
65+
66+
/**
67+
* Wire identity of a lowered tool. A `functions` child and an identical top-level declaration share
68+
* one identity because they denote the same logical tool: `buildTools` flattens the reserved group
69+
* without a namespace, so the parser already treats them as one and tolerates the duplicate.
70+
*/
71+
function loweredIdentity(namespace: string, name: string): string {
72+
return namespace === BUILTIN_FUNCTIONS_NAMESPACE
73+
? namespaceIdentity(BUILTIN_FUNCTIONS_NAMESPACE, name)
74+
: namespaceIdentity(namespace, name);
75+
}
76+
77+
function loweredWireName(namespace: string, name: string): string {
78+
return namespace === BUILTIN_FUNCTIONS_NAMESPACE ? name : namespacedToolName(namespace, name);
4779
}
4880

4981
function addSelector(
@@ -62,53 +94,48 @@ type NamespaceRewritePlan = {
6294
selectors: Map<string, string | null>;
6395
};
6496

97+
/** Two distinct logical tools would occupy one wire name; the caller maps this to a 400. */
98+
export class NamespaceToolCollisionError extends Error {}
99+
65100
function buildRewritePlan(groups: readonly unknown[][]): NamespaceRewritePlan {
66101
const aliases = new Map<string, RoutedNamespaceToolIdentity>();
67102
const identities = new Map<string, string>();
68103
const selectors = new Map<string, string | null>();
69-
const directNames = new Set<string>();
104+
const wireOwners = new Map<string, string>();
70105

71106
for (const group of groups) {
72107
for (const tool of group) {
73-
if (
74-
isPlainObject(tool)
75-
&& tool.type !== "namespace"
76-
&& typeof tool.name === "string"
77-
&& tool.name.length > 0
78-
) {
79-
directNames.add(tool.name);
108+
if (isPlainObject(tool) && tool.type !== "namespace" && isRepresentableName(tool.name)) {
109+
// A bare declaration is the reserved group's flattened form, so it claims that identity:
110+
// declaring the same tool both ways is the duplicate the parser already tolerates, not a
111+
// collision, and `promoteClientLoadedTools` produces exactly that shape.
112+
wireOwners.set(tool.name, loweredIdentity(BUILTIN_FUNCTIONS_NAMESPACE, tool.name));
80113
addSelector(selectors, tool.name, tool.name);
81114
}
82115
}
83116
}
84117

85-
const wireOwners = new Map<string, string>();
86-
for (const name of directNames) wireOwners.set(name, `direct:${name}`);
87-
88118
for (const group of groups) {
89119
for (const tool of group) {
90-
const parent = namespaceToolIdentity(tool);
91-
const children = namespaceChildren(tool);
92-
if (!parent || !children) continue;
93-
for (const child of children) {
120+
const parsed = parseNamespaceGroup(tool);
121+
if (!parsed) continue;
122+
for (const child of parsed.children) {
94123
const childName = child.name as string;
95-
const identity = namespaceIdentity(parent.namespace, childName);
96-
const wireName = parent.namespace === BUILTIN_FUNCTIONS_NAMESPACE
97-
? childName
98-
: namespacedToolName(parent.namespace, childName);
124+
const identity = loweredIdentity(parsed.namespace, childName);
125+
const wireName = loweredWireName(parsed.namespace, childName);
99126
const owner = wireOwners.get(wireName);
100127
if (owner !== undefined && owner !== identity) {
101-
throw new Error(
128+
throw new NamespaceToolCollisionError(
102129
`namespace tool wire-name collision for "${wireName}"; rename one of the colliding tools`,
103130
);
104131
}
105132
wireOwners.set(wireName, identity);
106133
identities.set(identity, wireName);
107134
addSelector(selectors, wireName, wireName);
108-
addSelector(selectors, `${parent.namespace}.${childName}`, wireName);
135+
addSelector(selectors, `${parsed.namespace}.${childName}`, wireName);
109136
addSelector(selectors, childName, wireName);
110-
if (parent.namespace !== BUILTIN_FUNCTIONS_NAMESPACE) {
111-
aliases.set(wireName, { namespace: parent.namespace, name: childName });
137+
if (parsed.namespace !== BUILTIN_FUNCTIONS_NAMESPACE) {
138+
aliases.set(wireName, { namespace: parsed.namespace, name: childName });
112139
}
113140
}
114141
}
@@ -117,72 +144,96 @@ function buildRewritePlan(groups: readonly unknown[][]): NamespaceRewritePlan {
117144
return { aliases, identities, selectors };
118145
}
119146

147+
/**
148+
* Lower every namespace group in one tool container. `emitted` is shared across the whole body so
149+
* a tool declared both bare and under `functions` is written once rather than twice.
150+
*
151+
* No `type: "namespace"` value survives this pass, including a group this layer cannot read:
152+
* relaying the private shape is what the strict gateway rejects.
153+
*/
120154
function rewriteToolList(
121155
tools: unknown[],
122156
plan: NamespaceRewritePlan,
123-
): { tools: unknown[]; changed: boolean } {
157+
emitted: Set<string>,
158+
): unknown[] {
124159
let changed = false;
125160
const rewritten: unknown[] = [];
126161
for (const tool of tools) {
127-
const parent = namespaceToolIdentity(tool);
128-
const children = namespaceChildren(tool);
129-
if (!parent || !children) {
130-
rewritten.push(tool);
162+
if (isPlainObject(tool) && tool.type === "namespace") {
163+
changed = true;
164+
const parsed = parseNamespaceGroup(tool);
165+
if (!parsed) continue;
166+
for (const child of parsed.children) {
167+
const wireName = plan.identities.get(loweredIdentity(parsed.namespace, child.name as string));
168+
if (wireName === undefined || emitted.has(wireName)) continue;
169+
emitted.add(wireName);
170+
rewritten.push(wireName === child.name ? child : { ...child, name: wireName });
171+
}
131172
continue;
132173
}
133-
changed = true;
134-
for (const child of children) {
135-
const identity = namespaceIdentity(parent.namespace, child.name as string);
136-
const wireName = plan.identities.get(identity);
137-
rewritten.push(wireName && wireName !== child.name ? { ...child, name: wireName } : child);
138-
}
174+
if (isPlainObject(tool) && isRepresentableName(tool.name)) emitted.add(tool.name);
175+
rewritten.push(tool);
139176
}
140-
return changed ? { tools: rewritten, changed: true } : { tools, changed: false };
177+
return changed ? rewritten : tools;
141178
}
142179

143-
function rewriteNamedSelector(value: unknown, plan: NamespaceRewritePlan): unknown {
180+
/**
181+
* Resolve one `{namespace?, name}` reference to its wire name and drop the private `namespace` key.
182+
*
183+
* `bareFallback` is for tool_choice, where a bare name is a selector the caller expects resolved
184+
* against the catalog. Replayed call items pass `false`: a history item records which tool actually
185+
* ran, so resolving a bare name through a same-named namespace child would rewrite history on a
186+
* coincidence rather than translate it.
187+
*
188+
* An explicit namespace is always lowered, even when this turn's catalog no longer declares that
189+
* group — a compaction turn drops the whole catalog, and a catalog can change mid-session. Leaving
190+
* the key in place ships a Codex-private field to a gateway that rejects unknown fields, which is
191+
* the failure this layer exists to prevent, and this layer's own response restoration is what put
192+
* the key on the item.
193+
*/
194+
function rewriteNamedSelector(
195+
value: unknown,
196+
plan: NamespaceRewritePlan,
197+
bareFallback: boolean,
198+
): unknown {
144199
if (!isPlainObject(value) || typeof value.name !== "string") return value;
145-
const explicitNamespace = typeof value.namespace === "string" ? value.namespace : undefined;
146-
const wireName = explicitNamespace
147-
? plan.identities.get(namespaceIdentity(explicitNamespace, value.name))
148-
: plan.selectors.get(value.name) ?? undefined;
149-
if (!wireName) return value;
150-
const { namespace: _namespace, ...rest } = value;
151-
return wireName === value.name && explicitNamespace === undefined
152-
? value
153-
: { ...rest, name: wireName };
200+
if (typeof value.namespace !== "string") {
201+
if (!bareFallback) return value;
202+
const wireName = plan.selectors.get(value.name) ?? undefined;
203+
return wireName === undefined || wireName === value.name ? value : { ...value, name: wireName };
204+
}
205+
const { namespace, ...rest } = value;
206+
const wireName = plan.identities.get(loweredIdentity(namespace, value.name))
207+
?? loweredWireName(namespace, value.name);
208+
return { ...rest, name: wireName };
154209
}
155210

156211
function rewriteToolChoice(value: unknown, plan: NamespaceRewritePlan): unknown {
157212
if (!isPlainObject(value)) return value;
158213
if ((value.type === "function" || value.type === "custom") && typeof value.name === "string") {
159-
return rewriteNamedSelector(value, plan);
214+
return rewriteNamedSelector(value, plan, true);
160215
}
161216
if (value.type !== "allowed_tools" || !Array.isArray(value.tools)) return value;
162217
let changed = false;
163218
const tools = value.tools.map(tool => {
164-
if (
165-
!isPlainObject(tool)
166-
|| (tool.type !== "function" && tool.type !== "custom")
167-
|| typeof tool.name !== "string"
168-
) return tool;
169-
const rewritten = rewriteNamedSelector(tool, plan);
219+
if (!isPlainObject(tool) || typeof tool.name !== "string") return tool;
220+
const rewritten = rewriteNamedSelector(tool, plan, true);
170221
changed ||= rewritten !== tool;
171222
return rewritten;
172223
});
173224
return changed ? { ...value, tools } : value;
174225
}
175226

176-
function rewriteInputItem(item: unknown, plan: NamespaceRewritePlan): unknown {
227+
function rewriteInputItem(item: unknown, plan: NamespaceRewritePlan, emitted: Set<string>): unknown {
177228
if (!isPlainObject(item)) return item;
178229
if (item.type === "additional_tools" && Array.isArray(item.tools)) {
179-
const rewritten = rewriteToolList(item.tools, plan);
180-
return rewritten.changed ? { ...item, tools: rewritten.tools } : item;
230+
const tools = rewriteToolList(item.tools, plan, emitted);
231+
return tools === item.tools ? item : { ...item, tools };
181232
}
182233
if (
183234
(item.type === "function_call" || item.type === "custom_tool_call")
184235
&& typeof item.name === "string"
185-
) return rewriteNamedSelector(item, plan);
236+
) return rewriteNamedSelector(item, plan, false);
186237
return item;
187238
}
188239

@@ -201,13 +252,23 @@ export function rewriteRoutedNamespaceToolsForUpstream(body: unknown): {
201252
if (!isPlainObject(body)) return { body, aliases: new Map() };
202253
const groups = collectResponsesToolGroups(body);
203254
const plan = buildRewritePlan(groups);
204-
if (plan.identities.size === 0) return { body, aliases: plan.aliases };
205255

206-
let tools = body.tools;
207-
if (Array.isArray(body.tools)) tools = rewriteToolList(body.tools, plan).tools;
256+
// Deliberately not gated on the plan being non-empty: a turn whose catalog is gone still replays
257+
// call items carrying a private `namespace`, and the routed compaction turn strips the whole tool
258+
// surface before this runs.
259+
const emitted = new Set<string>();
260+
const tools = Array.isArray(body.tools) ? rewriteToolList(body.tools, plan, emitted) : body.tools;
208261

209262
let input = body.input;
210-
if (Array.isArray(body.input)) input = body.input.map(item => rewriteInputItem(item, plan));
263+
if (Array.isArray(body.input)) {
264+
let inputChanged = false;
265+
const rewrittenInput = body.input.map(item => {
266+
const next = rewriteInputItem(item, plan, emitted);
267+
if (next !== item) inputChanged = true;
268+
return next;
269+
});
270+
if (inputChanged) input = rewrittenInput;
271+
}
211272

212273
const toolChoice = rewriteToolChoice(body.tool_choice, plan);
213274
return {

0 commit comments

Comments
 (0)