Skip to content

Commit 5f42a91

Browse files
easonLiangWorldedtecheasonliang28edelaunaeasonLiangWorldedtechdsh-agent
authored
feat(api): abort signal support for openai-native (#1291)
* feat(api): abort signal support for openai-native provider (completePrompt + createMessage) - completePrompt now uses a request-local signal merged from options.abortSignal and options.timeoutMs via mergeAbortSignalAndTimeout, no longer clobbering the streaming this.abortController; AbortError is rethrown as-is so callers can identify cancellations - createMessage paths (executeRequest and makeResponsesApiRequest fallback) bridge metadata.abortSignal into the internal controller using the Bedrock pattern (pre-aborted guard + { once: true } listener) Tests: abort signal passthrough, timeout abort, streaming-controller isolation, merged-signal abort, pre-aborted AbortError, fallback fetch pre-aborted/mid-request abort, non-Error rethrow, gpt-5.1 request-body coverage, response id/encrypted content accessors * feat(api): abort signal support for openai-compatible provider (completePrompt + createMessage) - completePrompt merges options.abortSignal and options.timeoutMs via mergeAbortSignalAndTimeout and forwards the merged signal to the AI SDK generateText abortSignal option - createMessage forwards metadata.abortSignal to streamText so in-flight streams are aborted on task cancellation Tests: new openai-compatible.spec.ts covering completePrompt signal/timeout passthrough, timeoutMs <= 0 disabled, pre-aborted AbortError, error propagation, and createMessage abortSignal bridging (pass-through, absent metadata, pre-aborted, mid-request abort) * fix(api): address CodeRabbit review on openai-native and openai-compatible abort handling - openai-native.ts: bridge external abort signal to a request-local controller in executeRequest and makeResponsesApiRequest; detach the { once: true } listener in finally so a late abort from an earlier request cannot cancel a later request's controller (listener closures no longer read the mutable this.abortController field) - openai-native.spec.ts: regression test - first stream completes normally, second stream runs with a different external signal, aborting the FIRST signal must not cancel the second stream - openai-compatible.spec.ts: timeout tests now assert the generated signal actually fires on its own ~50ms timeout (a never-expiring signal can no longer pass), the merged-signal timeout component fires independently of the caller signal, and caller signals pass through by identity when timeoutMs is 0 or negative * test(api): clear leftover guard timers in openai-compatible abort tests * refactor(api): adopt RequestConfigBuilder in feat/abort-r1-openai-native-compat abort wiring * test(api): cover primary-signal abort in builder merge regression tests * test(api): kill abort-signal mutation survivors in openai-native/openai-compatible Add focused abort-signal bridging tests for the bedrock-pattern bridge in OpenAiNativeHandler (executeRequest + makeResponsesApiRequest): once-only listener wiring and detachment, pre-aborted guard, mid-flight abort propagation, mid-stream break, late-abort listener detachment, and request-local controller ownership across concurrent SDK/fallback requests. Also assert the abortSignal property is left absent (not just undefined) in OpenAICompatibleHandler when no signal is supplied. Kills 26 of the 28 mutation-diff survivors on the PR diff; the two remaining OptionalChaining mutants (openai-native.ts L475/L719, externalAbortSignal?.removeEventListener) are unreachable variants: abortListener is only assigned when the signal is present, so optional chaining is behaviorally identical on every reachable path. * test(api): exclude proven-equivalent openai-native OptionalChaining mutants from the gate * fix(api): address reviewer abort-signal findings in openai-native and openai-compatible - openai-native: handleStreamResponse now checks the request-local controller before wrapping stream errors, so a user stop surfaces the contract AbortError ("The OpenAI Native request was aborted") exactly once instead of a wrapped error plus a double captureException. Regression tests: "should surface the contract AbortError once when the fallback stream read rejects on external abort" and "should not convert a non-abort stream error into an AbortError". - openai-native: extract the duplicated external-abort bridge from executeRequest and makeResponsesApiRequest into attachExternalAbort; the finally blocks call the returned cleanup (detach?.()), which removes the two Stryker OptionalChaining directives added in 118b921. All 11 abort-signal bridging specs pass unchanged. - request-config-builder: delete the unused mergeAbortSignals static and its three spec cases (zero production callers); mergeAbortSignalAndTimeout stays, with production callers in openai-native.ts and #1290's openai-codex.ts. - openai-native spec: replace the trivial "completePrompt should not replace an active streaming abort controller" assertion with the observable "should not let an earlier request's external abort affect a later request on the same handler" test. - openai-compatible spec: the pre-aborted completePrompt test now rejects with a real DOMException AbortError and asserts the exact message, since openai-compatible.ts passes SDK errors through without normalization. * test(api): reuse the open-stream fetch fixture and assert listener identity in openai-native specs * fix(api): per-request abort check in fallback SSE loop and release body on early exit * fix(api): normalize completePrompt SDK aborts to the contract AbortError * fix(api): classify SDK cancellations from the error alone, never from the signal * preflight synthetic head * test: kill catch-path mutants + document equivalent streamCompleted cleanup * preflight synthetic head * chore: re-trigger CI (flaky e2e-mock restart-persistence timeout) * test(abort-r1): assert the fallback request signal and the empty stream after cancellation * test(abort-r1): pin the fallback fetch to exactly one call * fix(abort-r1): hand back the contract AbortError on a mid-stream abort * fix(abort-r1): report the contract AbortError when the SDK iterator ends after an abort * test(providers): pin in-flight abort rejection for openai-compatible completePrompt CodeRabbit flagged that the completePrompt abort tests resolve the mock immediately, so the in-flight cancellation path was never observed. The new test keeps generateText pending, rejects with the real DOMException when the signal aborts, and asserts the provider's rejection after a mid-request abort - pinning the merged catch's abort guard (negative control: removing the guard turns this test red, the wrap would rename the error). 32/32 spec passes, eslint 0, prettier clean. * Address review: throw the contract AbortError at the SSE loop boundary The between-pulls abort check in handleStreamResponse broke out of the loop, so a cancellation landing after a chunk was handed out and before the next pull resolved createMessage with partial output instead of surfacing the cancellation. The check now throws the contract AbortError, mirroring the SDK path's own between-pulls checks. Two existing tests encoded the break behavior (empty/partial resolution after cancellation) and are updated to expect the contract AbortError; a new test pins cancellation between successive pulls with a body that hangs after one chunk (negative control verified: reverting the throw to break fails the new test). Verification at this head: 114/114 green in openai-native.spec.ts; tsc --noEmit 0 errors; eslint --max-warnings=0 clean on touched files; prettier clean. * fix(api): restore arrow glyphs in compat spec and type the native catch as unknown CodeRabbit follow-up on the main refresh: - openai-compatible.spec.ts: three comment arrows were mangled to '??' by an encoding roundtrip during the merge; restore '->'-style arrow glyph (the original '->' arrow character). - openai-native.ts: catch (sdkErr: unknown) instead of any; the body already handles unknown (isRequestAborted takes unknown and instanceof Error narrows before .name). The no-explicit-any suppression for the file returns to upstream's count of 27. * fix(api): reject non-finite and cap overflowing mergeAbortSignalAndTimeout delays Port of the #1300 timeout guard finding (CodeRabbit): Node's AbortSignal.timeout throws ERR_OUT_OF_RANGE for non-integer delays (Infinity) and clamps delays above 2^31-1 ms to 1 ms, so a non-finite timeoutMs threw a RangeError instead of running the request, and an overflowing one aborted almost immediately. The helper now treats non-finite values as "no timeout" and caps finite delays at 2^31-1 ms. Regression tests added; negative controls (reverting the guard) fail exactly the two new tests. * fix(api): round fractional timeouts before AbortSignal.timeout CodeRabbit follow-up on the timeout guard: Node validates AbortSignal.timeout delays as uint32, so finite fractions (1500.5, 0.5) threw ERR_OUT_OF_RANGE just like Infinity did. The helper now rounds delays up (a positive fraction never becomes 0 ms) before capping at 2^31-1 ms. Regression test mergeAbortSignalAndTimeout (undefined, 1.5); negative control (reverting the rounding) fails exactly that test. * test(api): assert the rounded delay reaching AbortSignal.timeout CodeRabbit: the fractional-timeout test only proved the call did not throw and eventually aborted - a round-down mutation (1.5 -> 1) would still pass. The test now spies AbortSignal.timeout and pins the received delay: 2 for 1.5 and 1 for 0.5 (never 0). Negative control: mutating Math.ceil to Math.floor fails exactly this test. * fix(providers): keep a status-carrying API failure that races an abort A 401/429 can land in the same tick the caller's signal aborts. Classifying purely by signal state turned those into cancellations: openai-compatible threw the raw error (losing the provider wrap) and openai-native replaced them with a synthetic AbortError DOMException (losing status and message), so retry/backoff and the UI never saw the provider failure. hasNumericHttpStatus distinguishes the two: an abort-shaped failure stays a cancellation, a status-carrying failure keeps its identity. createMessage's outer catch in openai-native also stops wrapping such errors. Also drops RequestConfigBuilder.mergeAbortSignalAndTimeout, the pass-through that duplicated the shared util: openai-native and openai-compatible now import mergeAbortSignalAndTimeout directly like bedrock and openai-codex. * fix(api): report a status-carrying stream failure before rethrowing it The hasNumericHttpStatus guard ran before TelemetryService.captureException, so a 401/429/5xx raised while reading the stream body kept its identity but lost its telemetry row. Capture first, then rethrow unchanged, and pin the ordering with a regression test (negative control: captureException called 0 times without the fix). Also covers the hasNumericHttpStatus boundary cases CodeRabbit asked for (Infinity, null, numeric fallback through statusCode). * test(api): stub the global fetch without adding a new explicit any The regression test used the file-wide �s any fetch idiom, which pushed the suppressed no-explicit-any count from 74 to 75. Suppression counts must never increase, so use vitest.stubGlobal instead. * fix(api): skip fallback work for a cancelled openai-native request Review follow-ups: the SDK catch path no longer issues the manual SSE fallback when this request other signal already aborted - a cancelled request must not start a second POST, even when the SDK failure was terminal rather than abort-shaped. Adds a focused test for that race (AbortError, fetch not called) and a regression test that completePrompt keeps its own request-local signal without touching an in-flight streaming request controller (the stream still owns handler.abortController and is unaffected by aborting the completion signal). openai-native + error-handler suites 153 passed; tsc clean for the touched files. * fix(openai-native): keep a status-carrying SDK error that races the caller abort Review follow-up: the no-fallback branch on an aborted request controller propagated only the generic cancellation, dropping the numeric HTTP status that retry, backoff and the UI read. A terminal SDK error with a numeric status is now propagated unchanged (the fallback still never runs); a status-less failure keeps the contract AbortError. Adds a 429 race test that asserts the original error and that fetch is never called. openai-native.spec.ts 118 passed; tsc clean. * test(openai-native): assert the SDK rate-limit error is propagated by reference Review follow-up: the terminal 429 race test compares the caught error with the injected rateLimited error by reference, so a re-wrapped or cloned error cannot pass. openai-native.spec.ts 118 passed. * test(openai-native): pin the full response-stream wrapper message * fix(openai-native): use createAbortError for the abort contract * test(openai-native): pin the rejection name in the status-less stream failure test CodeRabbit thread (root comment 4446908968): the test title promises a non-abort stream error never becomes an AbortError, but rejects.toThrow(string) matches the message by substring only - a regression that renamed the wrapped failure to AbortError while keeping the text would still pass. The test now captures the rejection and asserts toBeInstanceOf(Error), name === "Error" and the exact message. Negative control: renaming the wrapper at src/api/providers/openai-native.ts:837 to name = "AbortError" turns exactly this test red (AssertionError: expected 'AbortError' to be 'Error'); the capture style mirrors the neighbouring status-carrying-race test in the same file. Verified: parse-check OK, prettier --check clean, eslint --max-warnings=0 clean (openai-native.spec.ts suppression count unchanged at 74), vitest 125/125 in openai-native.spec.ts, tsc --noEmit 0 errors. * test(openai-native): keep the overlap test's second stream live during the abort The second request used asyncStreamFrom, which drains during the ticks, so the secondSdkSignal.aborted === false assertion was made after that request had already finished - a leak from the first request's controller would not have been observable. The second generator now waits on a gate that the test opens after the abort checks, and asserts the stream had not finished at that moment. Deleting the gate fails that new assertion with 'expected true to be false', which is the negative control. * fix(api): assert the stream failure by reference and drop equivalent guards - openai-native test: assert the SSE race failure reaches the caller as the same object (toBe), matching the SDK-path test. The previous object-shape assertions also passed for a wrapper that copies status and message. - openai-native: remove the streamCompleted guard. reader.cancel() on an already completed or failed stream resolves and never calls the underlying source's cancel, so both surviving mutants on that branch were equivalent and no test could kill them. - error-handler: drop the redundant typeof status === "number" check (Number.isFinite does not coerce, so it already rejects every non-number) and cover the object guard with a callable carrying a numeric status so the surviving mutant on that guard is killed. * test(api): name the pre-aborted fallback test after what it proves `should reject with AbortError when the external abortSignal is already aborted (fallback path)` asserted only the rejection. It now states the contract - no fallback POST is issued at all - and asserts fetch was never called, which is the behaviour the abort check exists for. Control: disabling the pre-fallback abort classification makes fetch run and the new assertion fail. openai-native.spec 125 passed, tsc --noEmit 0 errors. * fix(api): report a deadline as a timeout instead of a caller cancellation completePrompt normalised every SDK abort to the contract AbortError, so a request whose own timeout fired looked identical to a user stop. The merged signal keeps the TimeoutError reason when its deadline aborted first, so that case now throws createTimeoutError instead. openai-native.spec 125 passed, tsc --noEmit 0 errors, eslint --max-warnings=0 exit 0. * test(api): pin the deadline-versus-cancellation classification New test `completePrompt should report a deadline as TimeoutError rather than AbortError`: the SDK mock stays pending until the merged timeout signal aborts, then rejects with the SDK's own abort error - the shape the locked SDK produces. The provider must classify it as a timeout. Control: disabling the TimeoutError-reason branch makes the assertion fail with 'expected AbortError ... to match object { name: TimeoutError }', so the fix has a real red/green pair. openai-native.spec 126 passed, tsc --noEmit 0 errors, eslint --max-warnings=0 exit 0. * feat(api): carry numeric status on synthesized Responses API fallback errors The manual SSE fallback built a plain Error for non-2xx responses, so the numeric HTTP status that retry, backoff and the UI read was dropped for every synthesized provider failure. Attach response.status to the error so the catch path captures it once and rethrows it unchanged (row 4506478059). Test: forced fallback with a 418 response asserts status 418, the formatted message, and a single telemetry capture. * fix(api): capture abort-raced status-carrying SDK errors in telemetry The Responses API SDK path rethrew a terminal 401/429/5xx that landed in the same tick as an abort with its numeric status preserved, but skipped the provider telemetry capture the fallback path performs, so those failures vanished from error reporting. Capture the ApiProviderError once before propagating the original error object (row 4506478049). Test: the existing abort-race regression test now asserts a single telemetry capture carrying an ApiProviderError while the original error stays untouched. * fix(api): pass the caller timeout to the Responses API SDK as its request timeout completePrompt enforced timeoutMs only through the merged abort signal, so the SDK still aborted each attempt at the client default timeout: a completion allowed longer than the default could fail after retries, before the caller's deadline. Pass the normalized override as the SDK's timeout option as well (row 4506478073). getRequestTimeoutMs keeps the no-timeout contract (timeoutMs <= 0 / non-finite omits the option, since the SDK treats 0 as an immediate timeout). Test: the timeout trio's first case now asserts the SDK received timeout 50. * test(api): tighten completion cancellation assertions Two completion-path assertion gaps (rows 4506478027, 4506478090): the deadline test only matched the TimeoutError name, so a wrong phase string would still pass - it now asserts the full message; and no test covered completePrompt receiving the SDK's own AbortError with no options, where the request signal's reason is undefined - the new case locks the optional guard on the reason read. Controls: flipping the phase makes the message assertion fail, and dropping the optional guard turns the new case's rejection into a TypeError. * style(suppressions): reformat eslint-suppressions.json so format:check passes prettier --check . fails on this head with exactly one warn line, src/eslint-suppressions.json: the file lost its final newline and was serialised a wider way, so a 1326-entry table rendered as 33294 bytes instead of 30910. That is what eslint --prune-suppressions emits when it rewrites the file, and the merge carried that emission into the branch, which is why the PR shows +1326/-1326 on a file whose real change is a handful of counts. Reformatting only. Parsed contents are unchanged: the sum of all suppression counts is 2868 before and after, the table has 265 paths before and after, zero paths changed count, and the sorted-key canonical JSON is byte-identical between the two versions. Repo-wide prettier --check . now exits 0 with 'All matched files use Prettier code style!'. * test(openai-native): poll the observable instead of sleeping at three abort sites Three of the seven rule-banned fixed sleeps in the merged main spec become polls on named observables, each with an explicit timeout and a reason: 517 waits for the fetch double to have been called before aborting mid-request; 542 waits for the first request to register an in-flight stream; 559 waits for both, and the original length assertion is kept because polling replaces the sleep, not the assertion. An explicit timeout is what turns a condition that never fires into a failed test instead of a slow green. The rule (packages/config-eslint/api-provider-hygiene.js) names exactly one global, setTimeout, so a macrotask yield is not what it forbids; but three remaining sites use the banned API to do work the rule's message does not describe - a macrotask-phase yield an abort bridge depends on, a second immediate yield, and a hand-written race between a deadline and an abort event - and one stream-live poll resisted measurement here. No suppression was added: repo-wide eslint reports 4 banned-globals errors instead, eslint-suppressions.json still holds only its single no-explicit-any count of 74 for this path across 265 paths, and no-restricted-globals is absent from that file entirely. Measurement: focused spec under vitest.api.config.ts passes 128 of 128 in 2.48s against a 2.32s baseline on the unedited head - 0.16s, not a multiplication. * test(openai-native): drop the four fixed sleeps that trip the lint idiom rule The required compile arm runs `pnpm run lint` in src, and at 2a9c3d0 it exits 1 with 4 `no-restricted-globals` errors in src/api/providers/__tests__/openai-native.spec.ts (601:63, 609:20, 1263:36, 2037:35): "Unexpected use of 'setTimeout'. Poll with vi.waitFor on the observable condition. A fixed sleep either flakes or hides a missing await." The rule is apiSpecSleepConfig in packages/config-eslint/api-provider-hygiene.js, scoped to api/**/__tests__/**/*.spec.ts, which is why this file carries it and no other provider spec does; src/eslint-suppressions.json holds no entry for this rule, so these 4 errors were never suppressed. Site by site: - tick() has nothing observable to poll, so it is flushMacrotasks() from src/test-utils/timers, whose own header says to use it when there is no observable signal and to poll with vi.waitFor when there is one. - untilSignalAborted() is waiting for a deadline, so its fallback is now AbortSignal.timeout(timeoutMs) raced against the same abort event: same window, no timer to clear, nothing to forget on early abort. - The 429-races-abort row polls pullStarted (set when the reader enters pull and parks on the gate) and then flushes one macrotask, so the abort lands on a registered pending read instead of after 10 wall-clock ms. - The completePrompt abort row polls sdkRequestStarted, set inside the SDK double, so the abort arrives mid-request instead of after one timeout(0). A first cut polled untilSignalAborted with vi.waitFor(signal.aborted) at a 10 ms interval. That variant turned two abort-bridging rows red - "abort it mid-flight" (head :740) and "stop consuming the SDK stream" (head :788) - because the mocked stream's blocking window is load-bearing for those assertions: resolving it in ~10 ms lets the SDK iterator finish before the provider's between-pulls check runs. The deadline form is what is committed. Gates, run by hand from src with node v22.23.4: pnpm run lint exit 0 (was exit 1 with those 4 rows); eslint on this file exit 0, and with head content put back at the same path the same command exits 1 with exactly those 4 rows, so the change is what the control removes; vitest api shard 114 files, 4115 passed, 2 skipped (4117) exit 0; this file alone 128 passed; pnpm run check-types exit 0; prettier --check exit 0. eslint --prune-suppressions exits 0 and rewrites src/eslint-suppressions.json from 30,910 to 33,294 bytes with identical parsed content and 265 records both ways - the reformat 6e43f34 had just landed - so that file is reverted and is not in this commit. Test-only diff: __tests__ is in EXCLUDED_PATH_SEGMENTS of scripts/stryker-diff.mjs, so this delta adds no mutants and no production line moves. * fix(openai-native): gate the readiness waits, keep the status when the error body read fails Thread 4509766237 - the five readiness gates at spec 522/550/569/1271/2050 did not gate. vitest 4.1.12 keeps vi.waitFor live beside vi.waitUntil, but in dist/chunks/test.DNmyFkvJ.js waitFor hands a non-thenable result straight to onResolve and returns, so a predicate that evaluates false resolves the wait on the first check; waitUntil's onResolve starts with if (!result) return and polls until the timeout. The five sites now use vi.waitUntil with the predicates, timeouts and intervals unchanged, so the abort on the following line is placed mid-request as the comments already claimed. Thread 4509766269 - a body read that fails after the 429/503 headers arrived lost the numeric status, because the read at :765 sat outside the guarded parse and the outer catch had no status to recover. The read is guarded now: a cancelled read still surfaces as the contract AbortError, and a failed read carries the reason as the body text so the friendly status message and the status-carrying throw at the end of the block still apply. Control: the new 429 test is red without the guard (Tests 1 failed | 129 passed) and green with it (130 passed). Thread 4509766254 - spec 1542 now asserts the captured ApiProviderError's operation. Mutating the SDK-path operation literal to another value fails that assertion and nothing else (Tests 1 failed | 129 passed at :1542:53), which is what the old expect.any(ApiProviderError) could not do. * fix(openai-native): keep provider error text out of the abort-race telemetry capture The abort-race capture put the SDK's own message into ApiProviderError, and PostHogTelemetryClient forwards error.message to PostHog, so a 400 body that echoes request content (an address, a prompt fragment) could reach the telemetry backend. Report a fixed message plus the numeric status in errorCode instead - the client already turns errorCode into a structured property - and add a regression test whose provider error carries sensitive text. The propagated object is unchanged, so retry, backoff and the UI still read the provider's own error, and the capture still happens exactly once before the throw. * fix(openai-native): recheck the abort signal after a closed body, guard the abort-race capture Thread 4506478065 - the between-pulls check runs before reader.read(), so a done landing in the same turn as a cancellation ended the generator normally and handed the caller the partial stream the stop was meant to avoid. The done branch rechecks the request-local controller before breaking, and its own row drives exactly that interleaving. Thread 4515470580 - the capture sits between the status branch and the rethrow, so a throwing telemetry client would replace sdkErr and drop the status the retry reads. The site is guarded, and a row forces the capture to throw so the guard is not decorative; sweeping the other sites is left out because the unguarded form is repo-wide convention. Thread 4515470610 - getNumericHttpStatus gets its own describe: the statusCode-only fallback, status precedence, and the null, undefined, string, function and non-finite inputs, which is the gap the surviving mutants on error-handler.ts:64 and :68 describe. Thread 4509766262 - a fixture with three SSE events in one read tests the declared between-pulls bound rather than asserting it. Thread 4506478044 - the two request-local comments no longer claim the shared field is how a stop is delivered; comment-only, nothing executable moved. Plus the row the failed Regression Evidence check names: a fallback chunk is read, iterator.return() is called without aborting, and the open body must be cancelled with its reader lock released. --------- Co-authored-by: Eason Liang <[email protected]> Co-authored-by: edelauna <[email protected]> Co-authored-by: easonLiangWorldedtech <[email protected]> Co-authored-by: dsh-agent <dsh@local> Co-authored-by: easonLiangWorldedtech <[email protected]> Co-authored-by: f0-1528 <f0-1528@local>
1 parent f98fcf7 commit 5f42a91

4 files changed

Lines changed: 2430 additions & 88 deletions

File tree

0 commit comments

Comments
 (0)