Repository navigation
Commit ca05b73
feat(api): abort signal support for anthropic, anthropic-vertex, xai, minimax (#1293)
* feat(api): abort signal and timeout support for anthropic, anthropic-vertex, xai, minimax
- completePrompt: forward CompletePromptOptions abortSignal/timeoutMs into the SDK request for AnthropicHandler, AnthropicVertexHandler, XAIHandler, and MiniMaxHandler (request options built only when a signal/timeout is provided, preserving existing behavior)
- createMessage: bridge metadata?.abortSignal into a per-request AbortController using the Bedrock pattern (pre-aborted guard + { once: true } listener) and pass the internal signal as the SDK request signal; existing client-level timeout mechanisms are untouched
- xai: rethrow AbortError unmodified from createMessage/completePrompt so callers can detect error.name === 'AbortError'
- tests: port reference completePrompt signal/timeout propagation tests and add per-provider createMessage bridging tests (pre-aborted signal rejects with AbortError; mid-flight external abort cancels the request)
* ci: re-run e2e-mock on flaky restart-persistence (uses openrouter; this PR only touches anthropic family)
* fix(api): address CodeRabbit review on anthropic family abort handling
- anthropic.ts: use options?.timeoutMs !== undefined (was truthy) so a caller-supplied timeoutMs: 0 is forwarded to the SDK instead of silently dropped; all 4 family providers now share the same defined-check
- xai.ts: rethrow the OpenAI SDK's APIUserAbortError (exported from openai v5) unmodified from createMessage/completePrompt alongside native AbortError, since the SDK throws it when the request signal aborts and it would otherwise be mangled by handleOpenAIError
- tests: anthropic.spec.ts regression test asserting timeoutMs: 0 reaches the SDK as timeout: 0; xai.spec.ts exposes the real APIUserAbortError in the openai mock and asserts the SDK abort error surfaces as the same instance (unwrapped) through both createMessage and completePrompt
* fix(api): type xAI streaming request body with the OpenAI SDK type
- xai.ts: declare the streaming request body as OpenAI.Responses.ResponseCreateParamsStreaming instead of Record<string, any>, so the full request shape (incl. include/reasoning) is typechecked against the SDK and the as any on the create() call is no longer needed
- xai.ts: type the stream as AsyncIterable<OpenAI.Responses.ResponseStreamEvent> (matching the codebase pattern in mimo.ts/openai.ts) and drop the as unknown as AsyncIterable<any> double cast, since the SDK create() streaming overload already returns an AsyncIterable stream
- eslint-suppressions.json: reduce @typescript-eslint/no-explicit-any count for api/providers/xai.ts from 7 to 3 (four any usages removed)
* fix(api): detach external abort listeners and cover timeoutMs 0
- anthropic-vertex and xai createMessage retain the external abort listener callback and remove it in a finally block, so a long-lived caller signal no longer accumulates one listener (and closed-over controller) per completed request; specs assert the same callback is detached after stream completion.
- anthropic-vertex completePrompt spec adds the timeoutMs: 0 case asserting { timeout: 0 } reaches messages.create, guarding the defined-value check against a truthiness regression.
* refactor(api): map xai tool_choice to Responses API shape instead of casting
* fix(api): flatten xai allowed_tools entries for the Responses API
mapToolChoice forwarded Chat Completions allowlist entries
({ type: "function", function: { name } }) to responses.create, but the
Responses API expects { type: "function", name }. Flatten function-shaped
entries and leave other entry types untouched; add a regression test.
* test(api): cover abort-bridge and request-body mutation gaps
Add focused tests that kill the surviving and uncovered changed-code
mutants reported by the mutation-diff gate on this branch:
- xai: custom tool_choice mapping, allowed_tools passthrough for
unmappable entries, default-model max_output_tokens/temperature,
omitted tool parameters without tools, native AbortError passthrough
in completePrompt, exact Responses API request body, telemetry
capture for createMessage failures, and the once option on the
external abort listener.
- anthropic: a pending external signal does not pre-abort the bridged
signal, the external abort listener is registered with the once
option, and the default-model beta header is comma-joined.
- anthropic-vertex: message_start usage defaults outputTokens to zero
when output_tokens is omitted.
- minimax: completePrompt sends the configured temperature, user
message and stream flag; the abort bridge does not pre-abort pending
external signals and registers its listener with the once option.
Local mutation-diff gate: 7 surviving and 1 uncovered changed-code
mutants remain; all are unkillable without source changes (dead inner
default in anthropic.ts, spread-overridden stream flag and always-true
guards in xai.ts, single-element beta join and inert case label in
anthropic-vertex.ts).
* fix(api): remove dead code flagged by the mutation gate in anthropic, xai, and anthropic-vertex
- anthropic.ts: collapse the inner prompt-caching switch, whose 19 case
branches all executed an identical body plus an unreachable default,
into that body: push the prompt-caching beta, build the anthropic-beta
header, and attach the abort signal once before messages.create.
Pinned by anthropic.spec.ts (comma-joined beta header and
pending-abort-signal assertions). Deviation: tsconfig noImplicitReturns
forbids a string switch without a default, so the identical branches
were collapsed instead of deleting the default alone.
- xai.ts: drop the call-site `stream: true` spread override from
responses.create (the ResponseCreateParamsStreaming type requires the
flag in the request body, so the body literal keeps it and is what is
sent); collapse the model.maxTokens and model.temperature guards into
unconditional assignments; bind metadata?.tool_choice and
metadata?.parallelToolCalls once at the top of createMessage instead of
re-reading the metadata at the use sites. Pinned by xai.spec.ts exact
body assertions (stream: true, max_output_tokens 65_536, temperature 0,
tool_choice "auto", parallel_tool_calls true).
- anthropic-vertex.ts: send the single beta label directly (betas[0])
instead of building an array and joining it, and delete the no-op
content_block_stop case. Pinned by anthropic-vertex.spec.ts exact
anthropic-beta header assertions.
- xai.spec.ts: add a createMessage test for parallelToolCalls: false so
the defaulting of parallel_tool_calls stays pinned to an exact value.
* fix(api): attach anthropic abort signal via spread to drop the surviving guard mutant
* fix(api): detach external abort listeners when anthropic/minimax requests end
* test: assert non-cached Anthropic path rejects with AbortError on external abort
* Address review: shared abort bridge, canonical xAI aborts, beta join, cleanup coverage
- Extract the four copies of the external-abort bridge (anthropic,
anthropic-vertex, minimax, xai) into bridgeExternalAbortSignal in
utils/abort-signal.ts (single canonical comment, unit-tested incl. a
negative control that dispose detaches the exact retained callback).
- xai: normalize cancellations through isRequestAborted +
createAbortError in both createMessage and completePrompt; the raw
OpenAI SDK APIUserAbortError (name "Error", message "Request was
aborted.") no longer escapes to callers' AbortError guards.
- anthropic-vertex: send anthropic-beta as betas.join(",") again so a
second beta added to the builder later is not silently dropped.
- Move the xAI tool-choice mapping (mapToolChoice + typed request body
+ unconditional max_output_tokens/temperature) and the vertex
content_block_stop removal out of this abort-wiring unit per review:
the hunks return to a separate PR; xai.ts keeps its main-side lint
suppression count (7) accordingly.
- Pin listener cleanup beyond normal completion: vertex and xai specs
now cover create-rejects and consumer-early-return paths (vertex uses
an inline finally, so stream-cleanup coverage does not reach it).
Verification at this head: 273/273 green across the six touched specs;
tsc --noEmit 0 errors; eslint --max-warnings=0 clean on touched files;
prettier clean.
* chore(api): remove Stryker suppression comments from touched files
Per repo policy (AGENTS.md: Stryker suppression comments are banned;
the mutation gate rejects them in touched files). Equivalence
arguments for the resulting mutants are recorded in the PR body.
* fix(api): normalize stream-iteration aborts in the xAI responses path
CodeRabbit: an abort landing after responses.create resolved escaped
as the OpenAI SDK's APIUserAbortError (name "Error") from the yield*
over processResponsesApiStream, outside the create catch, so callers
classified the cancellation as a provider failure. Wrap the stream
iteration and normalize through isRequestAborted/createAbortError to
the series contract (name AbortError, "The xAI request was aborted").
Regression test pins the path; negative control: reverting the wrap
fails exactly that test (1 red / 37 green).
* fix(api): reject non-finite and cap overflowing mergeAbortSignalAndTimeout delays
Port of the #1300 timeout guard finding (CodeRabbit): Node's
AbortSignal.timeout throws ERR_OUT_OF_RANGE for non-integer delays
(Infinity) and clamps delays above 2^31-1 ms to 1 ms, so a
non-finite timeoutMs threw a RangeError instead of running the
request, and an overflowing one aborted almost immediately. The
helper now treats non-finite values as "no timeout" and caps
finite delays at 2^31-1 ms.
Regression tests added; negative controls (reverting the guard)
fail exactly the two new tests.
* fix(api): round fractional timeouts before AbortSignal.timeout
CodeRabbit follow-up on the timeout guard: Node validates
AbortSignal.timeout delays as uint32, so finite fractions (1500.5,
0.5) threw ERR_OUT_OF_RANGE just like Infinity did. The helper now
rounds delays up (a positive fraction never becomes 0 ms) before
capping at 2^31-1 ms. Regression test mergeAbortSignalAndTimeout
(undefined, 1.5); negative control (reverting the rounding) fails
exactly that test.
* test(api): assert the rounded delay reaching AbortSignal.timeout
CodeRabbit: the fractional-timeout test only proved the call did
not throw and eventually aborted - a round-down mutation (1.5 -> 1)
would still pass. The test now spies AbortSignal.timeout and pins
the received delay: 2 for 1.5 and 1 for 0.5 (never 0). Negative
control: mutating Math.ceil to Math.floor fails exactly this test.
* fix(anthropic-family): forward timeoutMs only when the SDK can take it
The four completePrompt copies forwarded any defined timeoutMs. The Anthropic SDK turns timeout: 0 into an immediate abort and its request validation throws on negative, NaN and Infinity, so the series convention (timeoutMs <= 0 or non-finite means no explicit timeout) must leave the option absent. Fractional delays are rounded up, matching mergeAbortSignalAndTimeout.
hasUsableTimeout is the shared predicate: it also backs mergeAbortSignalAndTimeout, so the rule is single-sourced for both the option and the signal.
* style: apply repo Prettier to the merge resolution
* refactor(api): share one SDK timeout conversion across the anthropic family
The four SDK option paths rounded fractional timeouts up but did not cap them, so an overflowing timeout would be clamped to 1 ms by Node. Add toSdkTimeoutMs (+ MAX_TIMEOUT_MS) next to hasUsableTimeout, use it in mergeAbortSignalAndTimeout and at all four provider sites, and cover the fractional/cap boundaries.
* style: apply repo Prettier to the timeout helper import
* test(providers): pin timeout rounding and the timer cap across the anthropic family
Review follow-ups: move the mergeAbortSignalAndTimeout JSDoc back above its function (MAX_TIMEOUT_MS and toSdkTimeoutMs now sit before it); the completePrompt timeout tests in the anthropic, anthropic-vertex, minimax and xai specs are now it.each cases covering a whole-number, a fractional (12.5 -> 13) and an over-cap (MAX+1 -> MAX) value, so forwarding the raw timeoutMs instead of calling toSdkTimeoutMs fails in every provider. Affected suites 364 passed.
* fix(api): lint request timeouts and spec loops, fix existing violations
---------
Co-authored-by: Eason Liang <[email protected]>
Co-authored-by: edelauna <[email protected]>
Co-authored-by: easonLiangWorldedtech <[email protected]>
Co-authored-by: dsh-agent <dsh@local>
Co-authored-by: Elliott de Launay <[email protected]>1 parent 8383de4 commit ca05b73
25 files changed
Lines changed: 2240 additions & 337 deletions
File tree
- packages/config-eslint
- src
- api
- providers
- __tests__
- utils
- __tests__
- transform/__tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
12 | 17 | | |
13 | 18 | | |
14 | 19 | | |
| |||
20 | 25 | | |
21 | 26 | | |
22 | 27 | | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
23 | 38 | | |
24 | 39 | | |
25 | 40 | | |
| |||
35 | 50 | | |
36 | 51 | | |
37 | 52 | | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| |||
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
0 commit comments