Skip to content

Commit 7dd145d

Browse files
committed
test(run-state): cover mixed turns and approval timestamps
1 parent e6674c0 commit 7dd145d

2 files changed

Lines changed: 38 additions & 4 deletions

File tree

‎docs/architecture/task-lifecycle-model.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -287,14 +287,14 @@ The checker applies every event except the seven turn events in every reachable
287287

288288
The checker also requires:
289289

290-
- the seven turn events (`turnStarted`, `firstChunkAwaitStarted`, `firstChunkAwaitEnded`, `toolRejected`, `toolUsed`, `toolFailed`, `streamReadCompleted`) to be accepted and to match the table in each of the 32 turn states. A shape is a group of reachable states with the same phase, stream, ask, latches and completion marker. The states in one shape also agree on whether a stream started, and whether the generation and the ask count are below their caps. The checker takes one state of each shape and gives it each of the 32 turn states. It applies every event to each of these states. Thus every other event must change the turn only as the table allows. The known-unsafe kernel runs use only two turn states: all false and all true. The search does not include the turn events, because they would make the state count 32 times larger;
290+
- the seven turn events (`turnStarted`, `firstChunkAwaitStarted`, `firstChunkAwaitEnded`, `toolRejected`, `toolUsed`, `toolFailed`, `streamReadCompleted`) to be accepted and to match the table in each of the 32 turn states. A shape is a group of reachable states with the same phase, stream, ask tag and timestamp, latches and completion marker. The states in one shape also agree on whether a stream started, and whether the generation and the ask count are below their caps. The checker takes one state of each shape and gives it each of the 32 turn states. It applies every event to each of these states. Thus every other event must change the turn only as the table allows. The known-unsafe kernel runs use only two turn states, all false and all true, except for a mixed-turn-only mutant that requires all 32. Separate mutants check that starting an ask preserves the turn after pending approvals with timestamps 1 and 2, before any stream or cancellation. The search does not include the turn events, because they would make the state count 32 times larger;
291291
- the event tags in `runState.ts` to equal the event tags in the table (once, at startup, as a text scan of `case` labels);
292292
- every event to be reachable, with the `askSettledStale` no-op, a settle for an ask that a newer pending ask replaced, as its own action;
293293
- every guard clause to be the only failing clause in some rejected event;
294294
- named witnesses, each tied to an event and the states around it: abort during a pending approval in a live stream, an ask after the stream ends, abandon after the stream ends, dispose after completion, a new stream after a failure cleanup without abort, a stream that starts without user input, an approval before the first stream, and a rejected late completion accept after a newer ask;
295295
- every ordered pair of the stream, ask, cancel, and dispose event sources to follow each other, where both events change the state;
296296
- no unseen successor at the depth bound;
297-
- thirty-nine known-unsafe kernels. The checker must reject each of them. They include a throwing kernel, a kernel that returns nothing and a kernel that returns both `ok` and `rejected`. There is also one kernel for each check that runs at every search step without the table; and
297+
- forty-two known-unsafe kernels. The checker must reject each of them. They include a throwing kernel, a kernel that returns nothing and a kernel that returns both `ok` and `rejected`. There is also one kernel for each check that runs at every search step without the table; and
298298
- self-tests that a small depth bound and a small state budget each fail the checker.
299299

300300
The kernel has no resume event. A resumed task is `initialized` followed by `streamStarted` or `askStarted`. The witnesses `first-stream-without-ask` and `approval-before-first-stream` cover those paths.

‎scripts/check-run-state-kernel.ts‎

Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -480,6 +480,7 @@ const shapeOf = ({ run, askCount }: ModelState) =>
480480
belowAskCap: askCount < MAX_ASK_TS,
481481
stream: run.stream.tag === "live" ? run.stream.cleanupFinished : "none",
482482
ask: run.ask.tag,
483+
askTs: run.ask.tag === "none" ? undefined : run.ask.askTs,
483484
latches: run.latches,
484485
lastCompletionAskTs: run.lastCompletionAskTs !== undefined,
485486
})
@@ -638,6 +639,7 @@ interface UnsafeKernel {
638639
step: Step
639640
caughtBy: RegExp
640641
useTable: boolean
642+
allTurnStates?: boolean
641643
}
642644

643645
const differsFor = (tag: EventTag) =>
@@ -849,6 +851,38 @@ const unsafeKernels: Record<string, UnsafeKernel> = {
849851
caughtBy: differsFor("askStarted"),
850852
useTable: true,
851853
},
854+
"ask-start-resets-mixed-turn": {
855+
step: (state, event) => {
856+
const result = nextRunState(state, event)
857+
return event.tag === "askStarted" && "ok" in result && state.turn.toolUsed && !state.turn.toolFailed
858+
? { ok: { ...result.ok, turn: { ...result.ok.turn, toolUsed: false } } }
859+
: result
860+
},
861+
caughtBy: differsFor("askStarted"),
862+
useTable: true,
863+
allTurnStates: true,
864+
},
865+
...Object.fromEntries(
866+
[1, 2].map((askTs): [string, UnsafeKernel] => [
867+
`ask-start-resets-turn-after-approval-${askTs}`,
868+
{
869+
step: (state, event) => {
870+
const result = nextRunState(state, event)
871+
return event.tag === "askStarted" &&
872+
"ok" in result &&
873+
state.ask.tag === "approval" &&
874+
state.ask.askTs === askTs &&
875+
state.generation === 0 &&
876+
!state.latches.abort &&
877+
state.latches.reason === undefined
878+
? { ok: { ...result.ok, turn: { ...result.ok.turn, toolUsed: false } } }
879+
: result
880+
},
881+
caughtBy: differsFor("askStarted"),
882+
useTable: true,
883+
},
884+
]),
885+
),
852886
"later-stream-start-resets-turn": {
853887
step: (state, event) => {
854888
const result = nextRunState(state, event)
@@ -973,8 +1007,8 @@ const unsafeKernels: Record<string, UnsafeKernel> = {
9731007
}
9741008

9751009
function runUnsafeKernels(): void {
976-
for (const [name, { step, caughtBy, useTable }] of Object.entries(unsafeKernels)) {
977-
const options = { step, useTable, maxDepth: MAX_DEPTH, maxStates: MAX_STATES, allTurnStates: false }
1010+
for (const [name, { step, caughtBy, useTable, allTurnStates = false }] of Object.entries(unsafeKernels)) {
1011+
const options = { step, useTable, maxDepth: MAX_DEPTH, maxStates: MAX_STATES, allTurnStates }
9781012
assert.throws(() => runModelCheck(options), caughtBy, `unsafe kernel not caught as expected: ${name}`)
9791013
}
9801014
}

0 commit comments

Comments
 (0)