Skip to content

Commit 7ef7b86

Browse files
donjor-agent[bot]rsi-pr-agent
authored andcommitted
fix(server): retire live work when provider sessions detach (#1734)
## Problem Deliberate provider teardown reports a provider failure, holds handoff continuations, and strands questions and native tools on dead sessions. Stop also leaves questions from already failed runs interactive. ## Change Publish owned interrupted terminals before closing subscribers, then retire the released session's live requests, tools, and nodes. Shared detach cleans only its thread; cutoff and runtime identity checks preserve replacement work. Stop also retires requests from ended runs and runless work. Cleanup faults cannot skip teardown; attachment-locked retries retain the original cutoff and runtime ownership. Inline and deferred detach share native teardown helpers. ## Scope and approval The maintainer authorized fixing these lifecycle bugs, recording proof, opening PRs, and merging after review and green checks. Closes #1731. Closes #1610. ## Verification ### Automated - [x] **Tests:** 146 detach/Stop cases, exit 0 - [x] **Isolation:** replacement work survives - [x] **Failure:** runtime errors still fail - [x] **Types/build:** server, exit 0 - [x] **Gate:** `vp run fork:ci`, exit 0 ### Live - [x] **Disconnect:** pending input clears - [x] **Handoff:** continuation auto-starts - [x] **Cleanup:** no waiting row remains - [x] **Stop:** question clears; run interrupts - [x] **Submit:** next prompt completes Before, a dead session's question remains selectable. https://github.com/user-attachments/assets/2c91dddd-bbd5-4107-bcd3-a45a0fa50b4e After, disconnect clears its question and interrupts the run. https://github.com/user-attachments/assets/9c6feaf7-075f-4aae-b125-9d0b2774745a Before, handoff holds the continuation. https://github.com/user-attachments/assets/463310c8-0ff9-4dda-99ac-7cee4c44be3d After, handoff automatically produces `FINAL_HANDOFF_OK` and clears the sidebar waiting state. https://github.com/user-attachments/assets/2c9be041-6456-4e79-a5d3-f4f5f5e266a2 After, actual client Stop clears pending input and interrupts the turn. https://github.com/user-attachments/assets/1dd7c874-f8ad-4607-a6b4-5ef0cebd5b72 After Stop, a new submission completes. ![Completed submission](https://github.com/user-attachments/assets/1d097f7e-064d-4eaf-95d8-ad8130af1fc6) Live build `529ebd95849` contains byte-identical detach source from final head `91beba566f5790b2d2db1028deb25e222fbefbdb`. Fork run 8 interrupted; run 9 completed with `FINAL_SUBMIT_OK`. Root disconnect run 15 interrupted; requests show zero pending. Handoff run 17 interrupted intentionally; continuation run 18 started automatically and completed. Historical prompts retain the Queued annotation; the current sidebar waiting state clears. Fault tests cover typed lookup failures, repeated retry failures, persistence faults, shared runtime survival, replacement isolation, ended-run and runless requests. All clips play normally: Stop 10s, disconnect 10s, handoff 11s; trailing idle is cut and the claim is captioned. Handoff uses `startFromOrigin: false` because the isolated fixture has no reachable remote. The service owns teardown for all providers and clients; shared and single-thread runtimes are covered by tests. Mobile, Electron, and remote connections were not exercised live. Codex, gpt-6.1-sol (high); independent review: Pi, zai/glm-5.3-flash. ## Fork trailers Squashes: - 624237b fix(server): retire live work when provider sessions detach (#1734) (#1734) - 5389223 fixup! fix(server): retire live work when provider sessions detach (#1734) (#1734) - 960b715 fix(server): retire live work when provider sessions detach (#1734) (#1734) - 654d522 fixup! fix(server): retire live work when provider sessions detach (#1734) (#1734) - 74d91fc fixup! fix(server): retire live work when provider sessions detach (#1734) (#1734) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
1 parent a958a4e commit 7ef7b86

6 files changed

Lines changed: 2141 additions & 105 deletions

‎apps/server/src/orchestration-v2/Orchestrator.ts‎

Lines changed: 52 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,10 @@ import { planThreadDeletion } from "./ThreadDeletion.ts";
169169
import { pullRequestWatchWakePrefix } from "./pullRequestWatch.ts";
170170
import { dispatchThreadIssueCommandFork } from "./ThreadIssues.fork.ts"; // fork-hook: github-issues/orchestrator-issues-import
171171
import { recordCheckoutRecoveryFork } from "./checkoutMove.fork.ts"; // fork-hook: zmux-estate/decider-recovery-record-import
172+
import {
173+
cancelStoppedRuntimeRequestsFork,
174+
finishedRunIdsFork,
175+
} from "./providerSessionDetach.fork.ts"; // fork-hook: upstream-fixes/stopped-request-import
172176
import {
173177
checkoutMoveMetadataFork,
174178
detachesCheckoutRecoveryInlineFork,
@@ -10569,6 +10573,8 @@ const makeOrchestrator = Effect.fn("orchestrationV2.Orchestrator.layer")(functio
1056910573
"subagent",
1057010574
"assistant_message",
1057110575
"reasoning",
10576+
"user_input_request", // fork-hook: upstream-fixes/interrupt-input-items
10577+
"approval_request", // fork-hook: upstream-fixes/interrupt-approval-items
1057210578
],
1057310579
turnItemStatuses: ["pending", "running", "waiting"],
1057410580
},
@@ -10830,6 +10836,8 @@ const makeOrchestrator = Effect.fn("orchestrationV2.Orchestrator.layer")(functio
1083010836
"subagent",
1083110837
"assistant_message",
1083210838
"reasoning",
10839+
"user_input_request", // fork-hook: upstream-fixes/interrupt-request-items
10840+
"approval_request", // fork-hook: upstream-fixes/interrupt-request-items
1083310841
],
1083410842
turnItemStatuses: ["pending", "running", "waiting"],
1083510843
},
@@ -10900,6 +10908,18 @@ const makeOrchestrator = Effect.fn("orchestrationV2.Orchestrator.layer")(functio
1090010908
});
1090110909

1090210910
const emitEvent = emit(events, command);
10911+
// fork-hook: upstream-fixes/interrupt-request-cleanup
10912+
// The client's Stop rides run.interrupt (holdQueue), so ended-run and
10913+
// runless request retirement cannot live only on thread.stop. Runs the
10914+
// target's own paths still handle stay live here and are untouched.
10915+
yield* cancelStoppedRuntimeRequestsFork({
10916+
threadId: command.threadId,
10917+
runIds: finishedRunIdsFork(projection.runs),
10918+
includeRunless: true,
10919+
projection,
10920+
now,
10921+
emit: emitEvent,
10922+
});
1090310923
const interruptRequestItem: OrchestrationV2TurnItem = {
1090410924
id: idAllocator.derive.runSignalTurnItem({
1090510925
runId: run.id,
@@ -11110,6 +11130,14 @@ const makeOrchestrator = Effect.fn("orchestrationV2.Orchestrator.layer")(functio
1111011130
),
1111111131
);
1111211132
if (Option.isNone(sessionOption)) {
11133+
// fork-hook: upstream-fixes/dead-run-request-cleanup
11134+
yield* cancelStoppedRuntimeRequestsFork({
11135+
threadId: command.threadId,
11136+
runIds: new Set([run.id]),
11137+
projection,
11138+
now,
11139+
emit: emitEvent,
11140+
});
1111311141
yield* emitEvent({
1111411142
type: "turn-item.updated",
1111511143
threadId: command.threadId,
@@ -11388,10 +11416,21 @@ const makeOrchestrator = Effect.fn("orchestrationV2.Orchestrator.layer")(functio
1138811416
effects: Ref.Ref<Array<PendingOrchestrationEffectV2>>,
1138911417
) {
1139011418
const projection = yield* projectionStore
11391-
.getThreadRecords(command.threadId, ["runs", "providerThreads", "turnItems", "subagents"], {
11392-
turnItemTypes: ["command_execution", "dynamic_tool", "subagent"],
11393-
turnItemStatuses: ["pending", "running", "waiting"],
11394-
})
11419+
.getThreadRecords(
11420+
command.threadId,
11421+
["runs", "providerThreads", "turnItems", "subagents", "nodes", "runtimeRequests"],
11422+
{
11423+
// fork-hook: upstream-fixes/stop-request-records
11424+
turnItemTypes: [
11425+
"command_execution",
11426+
"dynamic_tool",
11427+
"subagent",
11428+
"user_input_request",
11429+
"approval_request",
11430+
], // fork-hook: upstream-fixes/stop-request-items
11431+
turnItemStatuses: ["pending", "running", "waiting"],
11432+
},
11433+
)
1139511434
.pipe(
1139611435
Effect.mapError(
1139711436
(cause) => new OrchestratorProjectionError({ threadId: command.threadId, cause }),
@@ -11406,6 +11445,15 @@ const makeOrchestrator = Effect.fn("orchestrationV2.Orchestrator.layer")(functio
1140611445
)
1140711446
? Effect.void
1140811447
: Ref.update(effects, (existing) => [...existing, delegatedTasksStopEffect(command)]);
11448+
// fork-hook: upstream-fixes/stopped-thread-request-cleanup
11449+
yield* cancelStoppedRuntimeRequestsFork({
11450+
threadId: command.threadId,
11451+
runIds: finishedRunIdsFork(projection.runs),
11452+
includeRunless: true,
11453+
projection,
11454+
now: yield* DateTime.now,
11455+
emit: emit(events, command),
11456+
});
1140911457
const latestRun = projection.runs.findLast((run) => run.status !== "queued");
1141011458
const target =
1141111459
projection.runs.findLast(

0 commit comments

Comments
 (0)