Skip to content

Commit ff48ded

Browse files
PierrunoYTclaude
andcommitted
fix(task): fail closed on unreadable chains and repair disposed rollback
- Treat a missing descendant record as dead only when its task ID has no live owner in any provider (isDeadDelegationChain). - Refresh the delegation chain with TaskHistoryStore.refreshStrict during runtime recovery: only a missing file drops the cached record; an unreadable, malformed, or mismatched record throws and aborts recovery. - When disposal cancels delegation after the parent commit, release the parent from the deleted child through recoverDelegationParent and restore its pending action, without recreating a task on the disposed provider. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
1 parent 216dfd4 commit ff48ded

8 files changed

Lines changed: 255 additions & 8 deletions

File tree

‎docs/architecture/task-lifecycle-model.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,13 +51,13 @@ TLA+/PlusCal or Quint with TLC becomes a better fit when the lifecycle needs tem
5151
| Atomic event step | `atomicReadAndUpdate`, `atomicUpdatePair`, and per-parent delegation transition lock |
5252
| Event interleaving | Competing completion, cancellation, abandonment, and new delegation calls |
5353

54-
The model has three fixed task slots, enough to cover competing siblings and a nested parent-child-grandchild chain. It explores every reachable interleaving through depth 12, deduplicating canonical states. Runtime ownership is modeled separately from persisted status: an `owner-loss` fault can remove the live owner of an active or delegated child without changing its history record, matching process termination or session skip. Recovery is enabled only when the awaited delegation chain has no live owner and ends in an interrupted/completed task, a delegated task with no `awaitingChildId`, or a missing awaited record. Representative checks also exercise rejected operations that do not create a new state: a second concurrent delegation while the first child is active, stale completion after re-delegation, late completion after abandonment, completion after interruption, and nested completion. Named semantic landmarks require the graph to retain interrupted-child re-delegation, nested delegation, and owner-loss recovery even when the raw state total changes.
54+
The model has three fixed task slots, enough to cover competing siblings and a nested parent-child-grandchild chain. It explores every reachable interleaving through depth 12, deduplicating canonical states. Runtime ownership is modeled separately from persisted status: an `owner-loss` fault can remove the live owner of an active or delegated child without changing its history record, matching process termination or session skip. Recovery is enabled only when the awaited delegation chain has no live owner and ends in an interrupted/completed task, a delegated task with no `awaitingChildId`, or a missing awaited record whose task ID also has no live owner. Representative checks also exercise rejected operations that do not create a new state: a second concurrent delegation while the first child is active, stale completion after re-delegation, late completion after abandonment, completion after interruption, and nested completion. Named semantic landmarks require the graph to retain interrupted-child re-delegation, nested delegation, and owner-loss recovery even when the raw state total changes.
5555

5656
Production completion also accepts a recovery-compatible `active` parent that still awaits the returning child, then clears the stale pointers. Normal model transitions never create that intermediate state, so it is covered by a focused reducer test rather than admitted as a generally valid reachable state.
5757

5858
The `recover-active` action models startup repair after an active child loses its owner. `recoverDelegationParent` clears the parent's child pointers but interrupts it when an ancestor still awaits it; a top-level parent becomes active. The repair journal accepts both target statuses so a crash between the child and parent writes cannot break that ancestor link. Filesystem tests cover journal replay and subsequent re-delegation. Provider tests separately cover registration delayed by the recovery reservation, including disposal or cancellation before registration and preventing later scheduling.
5959

60-
For [#1624](https://github.com/Zoo-Code-Org/Zoo-Code/issues/1624), `recoverDeadDelegatedChild` repairs a nested delegated child whose descendant chain ends dead. Startup reconciliation and runtime re-delegation check provider-wide ownership while reserving registration through persistence; persisted delegated status alone is not evidence of a live session.
60+
For [#1624](https://github.com/Zoo-Code-Org/Zoo-Code/issues/1624), `recoverDeadDelegatedChild` repairs a nested delegated child whose descendant chain ends dead. Startup reconciliation and runtime re-delegation check provider-wide ownership while reserving registration through persistence; persisted delegated status alone is not evidence of a live session. Runtime re-delegation refreshes the chain with `TaskHistoryStore.refreshStrict`, so an unreadable or invalid descendant record aborts recovery instead of looking missing. If disposal cancels a delegation after the parent commit, rollback releases the parent through `recoverDelegationParent` and restores its pending action, without recreating a task on the disposed provider.
6161

6262
## Shared-store concurrency model
6363

‎src/__tests__/ClineProvider.delegation.spec.ts‎

Lines changed: 136 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ function makeStoreStub(
2121
) {
2222
return {
2323
invalidate: vi.fn().mockResolvedValue(undefined),
24+
refreshStrict: vi.fn().mockResolvedValue(undefined),
2425
atomicReadAndUpdate: vi.fn(async (_taskId: string, updater: (h: HistoryItem) => HistoryItem) => {
2526
updater(parentHistoryItem)
2627
return []
@@ -90,6 +91,64 @@ describe("ClineProvider.delegateParentAndOpenChild()", () => {
9091
expect(taskHistoryStore.atomicReadAndUpdate).not.toHaveBeenCalled()
9192
})
9293

94+
it("releases the committed parent when disposal cancels delegation after the commit", async () => {
95+
const pendingAction = {
96+
kind: "create_subtask" as const,
97+
actionId: "create-action",
98+
approvalText: "{}",
99+
mode: "code",
100+
message: "Do something",
101+
todos: [],
102+
}
103+
let current: HistoryItem = { ...parentHistoryItem, status: "active", pendingAction }
104+
const child = { taskId: "child-1", run: vi.fn().mockResolvedValue(undefined) }
105+
const createTaskWithHistoryItem = vi.fn()
106+
const deleteTaskWithId = vi.fn().mockResolvedValue(undefined)
107+
const provider = {
108+
_disposed: false,
109+
taskScheduler: new TaskScheduler(),
110+
emit: vi.fn(),
111+
getCurrentTask: vi.fn(() => makeParentTask()),
112+
removeClineFromStack: vi.fn().mockResolvedValue(undefined),
113+
createTask: vi.fn().mockResolvedValue(child),
114+
handleModeSwitch: vi.fn().mockResolvedValue(undefined),
115+
deleteTaskWithId,
116+
createTaskWithHistoryItem,
117+
log: vi.fn(),
118+
isViewLaunched: false,
119+
} as unknown as ClineProvider
120+
const parentTask = makeParentTask()
121+
vi.mocked(provider.getCurrentTask).mockReturnValue(parentTask)
122+
Reflect.set(provider, "taskHistoryStore", {
123+
invalidate: vi.fn().mockResolvedValue(undefined),
124+
get: vi.fn(() => current),
125+
atomicReadAndUpdate: vi.fn(async (_taskId: string, updater: (item: HistoryItem) => HistoryItem) => {
126+
current = updater(current)
127+
// Disposal starts right after the parent commit lands.
128+
Reflect.set(provider, "_disposed", true)
129+
return [current]
130+
}),
131+
})
132+
133+
await expect(
134+
ClineProvider.prototype.delegateParentAndOpenChild.call(provider, {
135+
parentTaskId: "parent-1",
136+
message: "Do something",
137+
initialTodos: [],
138+
mode: "code",
139+
pendingActionId: "create-action",
140+
}),
141+
).rejects.toThrow("disposed before child scheduling")
142+
143+
expect(child.run).not.toHaveBeenCalled()
144+
expect(deleteTaskWithId).toHaveBeenCalledWith("child-1", false)
145+
// The parent no longer awaits the deleted child and keeps its unresolved action.
146+
expect(current).toMatchObject({ status: "active", awaitingChildId: undefined, delegatedToId: undefined })
147+
expect(current.pendingAction).toEqual(pendingAction)
148+
// A disposed provider must not recreate the parent task.
149+
expect(createTaskWithHistoryItem).not.toHaveBeenCalled()
150+
})
151+
93152
it("clears a matching pending action when delegation commits", async () => {
94153
const pendingAction = {
95154
kind: "create_subtask" as const,
@@ -861,7 +920,7 @@ describe("ClineProvider.delegateParentAndOpenChild()", () => {
861920
return []
862921
}),
863922
})
864-
taskHistoryStore.invalidate.mockImplementation(async (id: string) => {
923+
taskHistoryStore.refreshStrict.mockImplementation(async (id: string) => {
865924
if (id === oldChildId) cache.set(id, refreshedChild)
866925
if (id === grandchildId) cache.set(id, refreshedGrandchild)
867926
})
@@ -877,11 +936,85 @@ describe("ClineProvider.delegateParentAndOpenChild()", () => {
877936

878937
await ClineProvider.prototype["recoverDeadAwaitedChild"].call(provider, parent, oldChildId)
879938

880-
expect(taskHistoryStore.invalidate).toHaveBeenCalledWith(oldChildId)
881-
expect(taskHistoryStore.invalidate).toHaveBeenCalledWith(grandchildId)
939+
expect(taskHistoryStore.refreshStrict).toHaveBeenCalledWith(oldChildId)
940+
expect(taskHistoryStore.refreshStrict).toHaveBeenCalledWith(grandchildId)
882941
expect(cache.get(oldChildId)).toMatchObject({ status: "interrupted", awaitingChildId: undefined })
883942
})
884943

944+
it("fails closed when a descendant record cannot be read during recovery", async () => {
945+
const oldChildId = "old-child"
946+
const grandchildId = "grandchild"
947+
const parent = {
948+
...parentHistoryItem,
949+
status: "delegated" as const,
950+
awaitingChildId: oldChildId,
951+
delegatedToId: oldChildId,
952+
}
953+
const child = {
954+
...parentHistoryItem,
955+
id: oldChildId,
956+
status: "delegated" as const,
957+
awaitingChildId: grandchildId,
958+
delegatedToId: grandchildId,
959+
}
960+
const taskHistoryStore = makeStoreStub({
961+
get: vi.fn((id: string) => (id === oldChildId ? child : undefined)),
962+
})
963+
taskHistoryStore.refreshStrict.mockImplementation(async (id: string) => {
964+
if (id === grandchildId) throw Object.assign(new Error("EACCES: permission denied"), { code: "EACCES" })
965+
})
966+
const provider = {
967+
taskRegistry: { hasRunning: vi.fn().mockReturnValue(false) },
968+
taskHistoryStore,
969+
log: vi.fn(),
970+
} as unknown as ClineProvider
971+
Object.assign(provider, {
972+
isTaskRunningInAnyProvider: ClineProvider.prototype["isTaskRunningInAnyProvider"],
973+
refreshDelegationChain: ClineProvider.prototype["refreshDelegationChain"],
974+
})
975+
976+
await expect(
977+
ClineProvider.prototype["recoverDeadAwaitedChild"].call(provider, parent, oldChildId),
978+
).rejects.toThrow("EACCES")
979+
expect(taskHistoryStore.atomicReadAndUpdate).not.toHaveBeenCalled()
980+
})
981+
982+
it("does not recover when a missing descendant record is still live in a provider", async () => {
983+
const oldChildId = "old-child"
984+
const grandchildId = "grandchild"
985+
const parent = {
986+
...parentHistoryItem,
987+
status: "delegated" as const,
988+
awaitingChildId: oldChildId,
989+
delegatedToId: oldChildId,
990+
}
991+
const child = {
992+
...parentHistoryItem,
993+
id: oldChildId,
994+
status: "delegated" as const,
995+
awaitingChildId: grandchildId,
996+
delegatedToId: grandchildId,
997+
}
998+
const taskHistoryStore = makeStoreStub({
999+
// The grandchild record is absent, but its task still runs.
1000+
get: vi.fn((id: string) => (id === oldChildId ? child : undefined)),
1001+
})
1002+
const provider = {
1003+
taskRegistry: { hasRunning: vi.fn((id: string) => id === grandchildId) },
1004+
taskHistoryStore,
1005+
log: vi.fn(),
1006+
} as unknown as ClineProvider
1007+
Object.assign(provider, {
1008+
isTaskRunningInAnyProvider: ClineProvider.prototype["isTaskRunningInAnyProvider"],
1009+
refreshDelegationChain: ClineProvider.prototype["refreshDelegationChain"],
1010+
})
1011+
1012+
await expect(
1013+
ClineProvider.prototype["recoverDeadAwaitedChild"].call(provider, parent, oldChildId),
1014+
).resolves.toMatchObject({ status: "delegated" })
1015+
expect(taskHistoryStore.atomicReadAndUpdate).not.toHaveBeenCalled()
1016+
})
1017+
8851018
it("rejects a missing awaited-child record without creating a child", async () => {
8861019
const oldChildId = "missing-child"
8871020
const parent = {

‎src/core/task-persistence/TaskHistoryStore.ts‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -818,6 +818,34 @@ export class TaskHistoryStore {
818818
})
819819
}
820820

821+
/**
822+
* Re-read a single task for an ownership decision. Unlike `invalidate()`, only a missing file
823+
* removes the cache entry; an unreadable, malformed, or mismatched record throws and keeps the
824+
* cached entry, so callers fail closed instead of treating the task as absent.
825+
*/
826+
async refreshStrict(taskId: string): Promise<void> {
827+
return this.withLock(async () => {
828+
const filePath = await this.getTaskFilePath(taskId)
829+
let raw: string
830+
try {
831+
raw = await fs.readFile(filePath, "utf8")
832+
} catch (error) {
833+
if (error && typeof error === "object" && "code" in error && error.code === "ENOENT") {
834+
this.cache.delete(taskId)
835+
this.taskFileMtimes.delete(taskId)
836+
return
837+
}
838+
throw error
839+
}
840+
const item: unknown = JSON.parse(raw)
841+
if (typeof item !== "object" || item === null || (item as { id?: unknown }).id !== taskId) {
842+
throw new Error(`Invalid task history record: ${filePath}`)
843+
}
844+
this.cache.set(taskId, item as HistoryItem)
845+
this.taskFileMtimes.delete(taskId)
846+
})
847+
}
848+
821849
/**
822850
* Clear all in-memory cache entries; a subsequent `reconcile()` repopulates them from task files.
823851
*/

‎src/core/task-persistence/__tests__/TaskHistoryStore.spec.ts‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -528,6 +528,49 @@ describe("TaskHistoryStore", () => {
528528
})
529529
})
530530

531+
describe("refreshStrict()", () => {
532+
it.each([
533+
["missing", undefined],
534+
["read-error", { code: "EISDIR" }],
535+
["malformed", { name: "SyntaxError" }],
536+
["other-task-id", { message: expect.stringContaining("Invalid task history record") }],
537+
] as const)("keeps the cached record unless the file is missing (%s)", async (scenario, error) => {
538+
await store.initialize()
539+
store.dispose() // Keep filesystem watcher reconciliation out of this explicit refresh test.
540+
const owner = makeHistoryItem({ id: "owner", status: "delegated", awaitingChildId: "child" })
541+
await store.upsert(owner)
542+
const filePath = path.join(tmpDir, "tasks", "owner", GlobalFileNames.historyItem)
543+
if (scenario === "malformed") {
544+
await fs.writeFile(filePath, "{")
545+
} else if (scenario === "other-task-id") {
546+
await fs.writeFile(filePath, JSON.stringify({ ...owner, id: "other-task" }))
547+
} else {
548+
await fs.unlink(filePath)
549+
if (scenario === "read-error") await fs.mkdir(filePath)
550+
}
551+
552+
if (error) {
553+
await expect(store.refreshStrict("owner")).rejects.toMatchObject(error)
554+
expect(store.get("owner")).toEqual(owner)
555+
} else {
556+
await expect(store.refreshStrict("owner")).resolves.toBeUndefined()
557+
expect(store.get("owner")).toBeUndefined()
558+
}
559+
})
560+
561+
it("re-reads a valid record from disk", async () => {
562+
await store.initialize()
563+
const item = makeHistoryItem({ id: "strict-task", tokensIn: 100 })
564+
await store.upsert(item)
565+
const filePath = path.join(tmpDir, "tasks", "strict-task", GlobalFileNames.historyItem)
566+
await fs.writeFile(filePath, JSON.stringify({ ...item, tokensIn: 999 }))
567+
568+
await store.refreshStrict("strict-task")
569+
570+
expect(store.get("strict-task")?.tokensIn).toBe(999)
571+
})
572+
})
573+
531574
describe("invalidateAll()", () => {
532575
it("waits for an in-flight write before clearing the cache", async () => {
533576
const onWrite = vi.fn().mockResolvedValue(undefined)

‎src/core/task-persistence/__tests__/taskLifecycle.spec.ts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,14 @@ describe("task lifecycle transitions", () => {
109109
).toBe(false)
110110
expect(isDeadDelegationChain({ ...child, status: "active" }, (id) => tasks.get(id))).toBe(false)
111111
expect(isDeadDelegationChain({ ...child, awaitingChildId: "missing" }, (id) => tasks.get(id))).toBe(true)
112+
// A missing record is not proof of death when its task still has a live owner.
113+
expect(
114+
isDeadDelegationChain(
115+
{ ...child, awaitingChildId: "missing" },
116+
(id) => tasks.get(id),
117+
(id) => id === "missing",
118+
),
119+
).toBe(false)
112120
expect(
113121
isDeadDelegationChain({ ...child, awaitingChildId: child.id }, (id) =>
114122
id === child.id ? child : undefined,

‎src/core/task-persistence/index.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ export {
2222
interruptDelegatedChild,
2323
isDeadDelegationChain,
2424
recoverDeadDelegatedChild,
25+
recoverDelegationParent,
2526
LifecycleTransitionError,
2627
type HistoryItemStatus,
2728
VALID_TASK_STATUS_TRANSITIONS,

‎src/core/task-persistence/taskLifecycle.ts‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -91,8 +91,10 @@ export function isDeadDelegationChain(
9191
if (current.status === "interrupted" || current.status === "completed") return true
9292
if (current.status !== "delegated") return false
9393
if (!current.awaitingChildId) return true
94-
current = getTask(current.awaitingChildId)
95-
if (!current) return true
94+
const awaitedId: string = current.awaitingChildId
95+
current = getTask(awaitedId)
96+
// A missing record is not proof of death: the descendant may still run elsewhere.
97+
if (!current) return !isTaskLive(awaitedId)
9698
}
9799
}
98100

‎src/core/webview/ClineProvider.ts‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,7 @@ import {
128128
delegateTaskToChild,
129129
isDeadDelegationChain,
130130
recoverDeadDelegatedChild,
131+
recoverDelegationParent,
131132
interruptDelegatedChild,
132133
} from "../task-persistence"
133134
import { readTaskMessages } from "../task-persistence/taskMessages"
@@ -3835,7 +3836,8 @@ export class ClineProvider
38353836
let currentTaskId: string | undefined = taskId
38363837
while (currentTaskId && !visited.has(currentTaskId)) {
38373838
visited.add(currentTaskId)
3838-
await this.taskHistoryStore.invalidate(currentTaskId)
3839+
// Strict: an unreadable descendant must abort recovery, not look missing (and so dead).
3840+
await this.taskHistoryStore.refreshStrict(currentTaskId)
38393841
const current = this.taskHistoryStore.get(currentTaskId)
38403842
currentTaskId = current?.status === "delegated" ? current.awaitingChildId : undefined
38413843
}
@@ -4083,6 +4085,10 @@ export class ClineProvider
40834085
// synchronously under the store lock) so a concurrent abandon or completion cannot
40844086
// slip between the status snapshot and the write. An active child must never be
40854087
// silently detached.
4088+
// Set once the parent record durably awaits this child, so rollback can undo that link
4089+
// even when disposal cancels delegation after the commit.
4090+
let parentCommitted = false
4091+
let pendingActionBeforeCommit: HistoryItem["pendingAction"]
40864092
try {
40874093
if (this._disposed) {
40884094
throw new Error("[delegateParentAndOpenChild] Provider was disposed before delegation commit")
@@ -4097,12 +4103,14 @@ export class ClineProvider
40974103
? this.taskHistoryStore.get(historyItem.awaitingChildId)?.status
40984104
: undefined
40994105
const delegated = delegateTaskToChild(historyItem, child.taskId, awaitedChildStatus)
4106+
pendingActionBeforeCommit = historyItem.pendingAction
41004107
return {
41014108
...delegated,
41024109
pendingAction:
41034110
delegated.pendingAction?.actionId === pendingActionId ? undefined : delegated.pendingAction,
41044111
}
41054112
})
4113+
parentCommitted = true
41064114
if (this._disposed) {
41074115
throw new Error("[delegateParentAndOpenChild] Provider was disposed before child scheduling")
41084116
}
@@ -4141,6 +4149,30 @@ export class ClineProvider
41414149
}`,
41424150
)
41434151
}
4152+
if (parentCommitted) {
4153+
// Undo the committed link even on a disposed provider: otherwise the persisted parent
4154+
// awaits a deleted child until a later reconciliation repairs it.
4155+
try {
4156+
await this.taskHistoryStore.atomicReadAndUpdate(parentTaskId, (current) => {
4157+
if (current.status !== "delegated" || current.awaitingChildId !== child.taskId) {
4158+
return current // A newer transition already owns the parent.
4159+
}
4160+
const ancestor = current.parentTaskId
4161+
? this.taskHistoryStore.get(current.parentTaskId)
4162+
: undefined
4163+
return {
4164+
...recoverDelegationParent(current, ancestor),
4165+
pendingAction: pendingActionBeforeCommit,
4166+
}
4167+
})
4168+
} catch (repairError) {
4169+
this.log(
4170+
`[delegateParentAndOpenChild] Failed to release parent ${parentTaskId} from deleted child ${child.taskId}: ${
4171+
(repairError as Error)?.message ?? String(repairError)
4172+
}`,
4173+
)
4174+
}
4175+
}
41444176
if (!this._disposed) {
41454177
try {
41464178
const { historyItem: parentHistory } = await this.getTaskWithId(parentTaskId)

0 commit comments

Comments
 (0)