Skip to content

Commit aca5b31

Browse files
fix(api): race requesty request creation against its cancellation scope
createMessage awaited the SDK create() call directly at requesty.ts:242 while the same file documents, at 377-382, that [email protected] does not observe the signal inside retryRequest - it sleeps for the backoff, or a server-provided retry-after-ms of up to about a minute, before the next attempt. completePrompt races that await (383-385) and fetchModel is raced too (215, 346); request creation was the provider's only unraced SDK await, so an abort during a retry backoff left createMessage pending for the length of the sleep. Wrap it in the same rejectOnAbort race and add a row whose mockCreate never settles, which is the shape that hung before this change: with the two production lines reverted the new row fails with 'settle guard timed out after 500ms' and the other 76 pass.
1 parent 39202fe commit aca5b31

2 files changed

Lines changed: 40 additions & 1 deletion

File tree

‎src/api/providers/__tests__/requesty.spec.ts‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1169,6 +1169,38 @@ describe("RequestyHandler", () => {
11691169
})
11701170
})
11711171

1172+
it("rejects with AbortError when the SDK call never settles behind its retry backoff", async () => {
1173+
const handler = new RequestyHandler(mockOptions)
1174+
const controller = new AbortController()
1175+
1176+
// [email protected] sleeps through the retry backoff (or a server retry-after-ms) without
1177+
// observing the signal, so the promise create() returns can still be pending a minute
1178+
// after the caller cancelled. Emulate exactly that: a create() that was handed the
1179+
// request signal and never settles.
1180+
let notifyCreateStarted!: () => void
1181+
const createStarted = new Promise<void>((resolve) => {
1182+
notifyCreateStarted = resolve
1183+
})
1184+
mockCreate.mockImplementationOnce(() => {
1185+
notifyCreateStarted()
1186+
return new Promise<never>(() => {})
1187+
})
1188+
1189+
const metadata = makeCreateMessageMetadata({ abortSignal: controller.signal })
1190+
const generator = handler.createMessage("sys", [{ role: "user", content: "hi" }], metadata)
1191+
const nextPromise = generator.next()
1192+
void nextPromise.catch(() => {})
1193+
await withSettleGuard(createStarted)
1194+
controller.abort()
1195+
1196+
// Without the race at requesty.ts the awaited SDK promise stays pending and this guard
1197+
// rejects with its own timeout instead of the provider AbortError.
1198+
await expect(withSettleGuard(nextPromise)).rejects.toMatchObject({
1199+
name: "AbortError",
1200+
message: "The Requesty request was aborted",
1201+
})
1202+
})
1203+
11721204
it("rethrows non-abort creation errors from createMessage", async () => {
11731205
const handler = new RequestyHandler(mockOptions)
11741206
mockCreate.mockImplementationOnce(async () => {

‎src/api/providers/requesty.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -239,7 +239,14 @@ export class RequestyHandler extends BaseProvider implements SingleCompletionHan
239239
let stream
240240
try {
241241
// With streaming params type, SDK returns an async iterable stream
242-
stream = await this.client.chat.completions.create(completionParams, { signal: requestSignal })
242+
const pendingStream = this.client.chat.completions.create(completionParams, { signal: requestSignal })
243+
// Race the creation against the cancellation scope, as completePrompt does below:
244+
// [email protected] does not observe the signal inside `retryRequest`, which sleeps for the
245+
// backoff (or a server-provided `retry-after-ms`, up to about a minute) before issuing
246+
// the next attempt, so awaiting the SDK call alone can hold createMessage open long
247+
// after the caller cancelled. The race settles at once, and a rejection the SDK wins
248+
// first is still normalized by the catch below.
249+
stream = await rejectOnAbort(pendingStream, requestSignal, this.providerName)
243250
} catch (error) {
244251
// Aborted requests are user-initiated: surface them as AbortError instead of
245252
// a completion error.

0 commit comments

Comments
 (0)