Skip to content

Version Packages#4

Merged
threepointone merged 1 commit into
mainfrom
changeset-release/main
Feb 21, 2025
Merged

Version Packages#4
threepointone merged 1 commit into
mainfrom
changeset-release/main

Conversation

@threepointone

Copy link
Copy Markdown
Contributor

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.

Releases

@cloudflare/[email protected]

Patch Changes

@threepointone threepointone merged commit 4189ff4 into main Feb 21, 2025
@threepointone threepointone deleted the changeset-release/main branch February 21, 2025 00:24
@claude claude Bot mentioned this pull request Nov 4, 2025
threepointone added a commit that referenced this pull request Jun 1, 2026
…t recovery + per-turn timeout override

Follow-up to the stall-watchdog→bounded-recovery commit, addressing PR review.

Bug: stall-recovery budget exhaustion bypassed the recovery exhaustion path.
  When `_routeStallToBoundedRecovery` ran out of budget it returned `false` and
  both stream catches fell through to the GENERIC terminal path. That skipped
  `_exhaustChatRecovery`, so `onExhausted` never fired, the
  `chat:recovery:exhausted` event was not emitted, the submission was not marked
  interrupted, and the configured `terminalMessage` was not shown — the user saw
  the raw "Chat stream stalled…" error. This diverged from deploy-recovery
  exhaustion, contradicting the "same bounded path" goal.

  `_routeStallToBoundedRecovery` now returns `"scheduled" | "exhausted" |
  "disabled"`. On `"exhausted"` it routes through the SAME `_exhaustChatRecovery`
  path as deploy recovery (passing the partial + streamId + `firstSeenAt` as the
  turn-start proxy); both `_streamResult` and `_streamResultToRpcCallback`
  finalize the stream and return without re-running the generic terminal. Only
  `"disabled"` (chatRecovery off) falls through to the unchanged terminal path.

Cleanup: fixed an orphaned JSDoc block — the method had been inserted between
  `_rescheduleRecoveryAfterStableTimeout`'s docstring and its definition; moved
  it next to `_exhaustChatRecovery` so the docstring reattaches.

Feature (review #3): added a per-turn `TurnConfig.chatStreamStallTimeoutMs`
  override (returned from `beforeTurn`). Resolved in `_runInferenceLoop` and
  stashed on `_activeStallTimeoutMs`, read by both stream loops; an explicit `0`
  disables the watchdog for that turn; it auto-resets each turn. Lets a turn with
  a known-slow tool raise/disable the watchdog without permanently widening the
  instance-level window.

Tests:
  - `testStallRouteExhaustion` (surgical, deterministic): seeds an incident at
    the budget edge and routes one stall — asserts `onExhausted` fires once with
    `max_attempts_exceeded`, the incident is sealed `exhausted`, and the client
    sees the configured `terminalMessage` (not the raw stall error).
  - per-turn override test: instance watchdog off, per-turn override arms it, a
    stall still routes into recovery and the continuation completes.

Docs: think/index.md (config row + Chat Recovery section), lifecycle-hooks.md
  (TurnConfig table), observability.md (`chat:stream:stalled` now routes to
  recovery), README, and the changeset.

Note: review #4 (the RPC `StreamCallback` is abandoned — no onDone/onError — on a
  recovering/exhausting stall) is intentionally left as-is for now; the only
  strictly-correct fix is a new optional `onInterrupted()` signal, tracked as a
  follow-up.
threepointone added a commit that referenced this pull request Jun 1, 2026
…#1626) (#1643)

* fix(think): route a stream-stall watchdog abort into bounded recovery (#1626)

When `chatStreamStallTimeoutMs` is set and the inactivity watchdog fires on a
hung model/transport stream, the turn was failed terminally. A stall is just
another interruption — like a deploy or eviction — so it is now routed into the
SAME bounded chat-recovery path: the settled partial is preserved, a
continuation is scheduled (`_chatRecoveryContinue`), and the turn resumes,
reusing the existing incident budget (maxAttempts + wall-clock no-progress
window + progress-aware reset).

A transient hang (the common cause under deploy churn) recovers automatically;
a persistently hanging provider still terminalizes once the budget is exhausted
(the watchdog's original "kill the infinite spinner" guarantee, after bounded
retries). Automatic whenever the watchdog is enabled and chatRecovery is on
(the Think default) — no new config; with chatRecovery off, a stall stays
terminal as before. Idempotency matches deploy recovery (settled tool results
durable; a mid-execution tool re-runs on the continuation).

Implementation:
- `_iterateWithStallWatchdog` now throws a distinct `ChatStreamStalledError`
  so the read-loop catch can tell a stall from an in-band model error (only the
  stall routes to recovery).
- New `_routeStallToBoundedRecovery` opens/advances a recovery incident and
  schedules the continuation (or returns false when the budget is exhausted,
  so the caller falls through to the existing terminal path). Gated on
  chatRecovery being enabled.
- Both stream paths (`_streamResult`, `_streamResultToRpcCallback`) branch their
  catch: persist the settled partial (re-anchor), route to recovery, and
  suppress the terminal error when a continuation is scheduled.

Tests:
- Unit: a stall on the first inference recovers via the scheduled continuation
  (no terminal error); the existing watchdog→terminal test now runs with
  recovery off (the no-recovery gate).
- E2E (real wrangler dev, no process kill — a stall is an in-isolate hang): the
  watchdog fires, the real alarm fires the continuation, and the turn completes
  with the continuation's output and no orphaned fiber rows.

Think-only (the watchdog is Think-only; AIChatAgent delegates the read loop).

Co-authored-by: Cursor <[email protected]>

* fix(think): address #1626 review — route stall exhaustion through chat recovery + per-turn timeout override

Follow-up to the stall-watchdog→bounded-recovery commit, addressing PR review.

Bug: stall-recovery budget exhaustion bypassed the recovery exhaustion path.
  When `_routeStallToBoundedRecovery` ran out of budget it returned `false` and
  both stream catches fell through to the GENERIC terminal path. That skipped
  `_exhaustChatRecovery`, so `onExhausted` never fired, the
  `chat:recovery:exhausted` event was not emitted, the submission was not marked
  interrupted, and the configured `terminalMessage` was not shown — the user saw
  the raw "Chat stream stalled…" error. This diverged from deploy-recovery
  exhaustion, contradicting the "same bounded path" goal.

  `_routeStallToBoundedRecovery` now returns `"scheduled" | "exhausted" |
  "disabled"`. On `"exhausted"` it routes through the SAME `_exhaustChatRecovery`
  path as deploy recovery (passing the partial + streamId + `firstSeenAt` as the
  turn-start proxy); both `_streamResult` and `_streamResultToRpcCallback`
  finalize the stream and return without re-running the generic terminal. Only
  `"disabled"` (chatRecovery off) falls through to the unchanged terminal path.

Cleanup: fixed an orphaned JSDoc block — the method had been inserted between
  `_rescheduleRecoveryAfterStableTimeout`'s docstring and its definition; moved
  it next to `_exhaustChatRecovery` so the docstring reattaches.

Feature (review #3): added a per-turn `TurnConfig.chatStreamStallTimeoutMs`
  override (returned from `beforeTurn`). Resolved in `_runInferenceLoop` and
  stashed on `_activeStallTimeoutMs`, read by both stream loops; an explicit `0`
  disables the watchdog for that turn; it auto-resets each turn. Lets a turn with
  a known-slow tool raise/disable the watchdog without permanently widening the
  instance-level window.

Tests:
  - `testStallRouteExhaustion` (surgical, deterministic): seeds an incident at
    the budget edge and routes one stall — asserts `onExhausted` fires once with
    `max_attempts_exceeded`, the incident is sealed `exhausted`, and the client
    sees the configured `terminalMessage` (not the raw stall error).
  - per-turn override test: instance watchdog off, per-turn override arms it, a
    stall still routes into recovery and the continuation completes.

Docs: think/index.md (config row + Chat Recovery section), lifecycle-hooks.md
  (TurnConfig table), observability.md (`chat:stream:stalled` now routes to
  recovery), README, and the changeset.

Note: review #4 (the RPC `StreamCallback` is abandoned — no onDone/onError — on a
  recovering/exhausting stall) is intentionally left as-is for now; the only
  strictly-correct fix is a new optional `onInterrupted()` signal, tracked as a
  follow-up.

---------

Co-authored-by: Cursor <[email protected]>
threepointone added a commit that referenced this pull request Jun 20, 2026
…k to RFC

Consolidate the scattered "Still open" notes into one orientation section near
the top of the RFC so a fresh working session has the full map in one place:
what has landed (with commits), the prioritized open items (real Workers AI
provider run; API-ergonomics findings #3/#4; start-id alignment; Phase 6/7), and
what is explicitly deferred / post-v1. Doc-only.

Co-authored-by: Cursor <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant