Skip to content

perf(graph): work-conserving slot scheduler replaces barrier batching - #12

Merged
momozi1996 merged 1 commit into
momozi1996:mainfrom
xxwtiancai:perf/graph-slot-scheduler
Oct 27, 2026
Merged

momozi1996 merged 1 commit into
momozi1996:mainfrom
xxwtiancai:perf/graph-slot-scheduler

Conversation

@xxwtiancai

Copy link
Copy Markdown
Contributor

Problem

executeGraph scheduled ready nodes in barrier batches: readyNodes().slice(0, concurrency) followed by await Promise.all(batch). The slowest node of every batch gated the dispatch of the next batch. LLM subagent durations are wildly heterogeneous (a node with maxRetries=2 can legitimately run 3× a sibling's wall time), so:

  • fast nodes finish and their slots sit idle while unrelated stragglers run out the clock;
  • dependents of quick parents wait behind slow siblings they don't depend on.

Makespan degenerated into Σ per-batch maxima instead of tracking the concurrency-limited critical path. Worst case: level 1 = [5s node, 300s node], a node depending only on the 5s one still waits 300s.

Change

Replace the batch loop with a work-conserving scheduler:

  • Up to concurrency nodes stay in flight; every completion immediately re-routes dynamic edges, settles skips, and refills the freed slot.
  • All scheduling decisions flow through a single serialized pump() chain (completions enqueue onto it), so concurrent completions can never double-dispatch a node.
  • New interleaving handled explicitly: under slot scheduling a rework/conditional route can target a node still executing (impossible under barriers). Those activations are deferred (deferredActivations) and applied when the target lands; the rework budget is re-checked at activation time so two failures deferring onto the same fixer cannot jointly overspend it.
  • Phase ordering per pump pass is identical to the old loop (route evaluation → approval pause → skip propagation → dispatch), so approval pauses, rework loops, skip settlement, the execution budget rail, persistence points, and onNode ordering all behave exactly as before — all 8 existing graph-execution tests pass unmodified.

Benchmark

Same DAG under both schedulers (concurrency: 2): one 3s straggler independent of a 100ms → 1s → 1s chain.

scheduler run 1 run 2
barrier (main) 4244 ms 4236 ms
slots (this PR) 3048 ms 3053 ms

−28% makespan on a 4-node graph; the gain grows with graph width and duration variance (real LLM graphs vary 10s–300s per node). Reproduce: fixture child that sleeps N ms via Atomics.wait, driven through the real spawnSubagent path with process.argv[1] pointed at it (same technique as the existing tests).

Regression test

New test pins the scheduling property, not a timing constant: a node depending only on a quick sibling must start while the straggler is still in flight (startedAt(after) + 800 < finishedAt(slow)), with the dependency edge still respected causally. The 800 ms margin keeps it robust on loaded CI runners.

Verification

  • Full suite: 500/500 pass (npx tsx --test src/test/*.test.ts), tsc --noEmit clean
  • All 8 pre-existing graph-execution tests pass unmodified (approval pauses, rework loops, skip propagation, resume, budget rail)

executeGraph ran ready nodes in barrier batches: Promise.all over each
batch, so the slowest node of every batch gated the dispatch of the next.
LLM subagents have wildly heterogeneous durations (retries can triple a
node's wall time), so fast nodes finished and left slots idle while
dependents of quick parents waited behind unrelated stragglers. Makespan
degenerated into the sum of per-batch maxima instead of tracking the
concurrency-limited critical path.

Replace the batch loop with a work-conserving scheduler: up to
concurrency nodes stay in flight and every completion immediately
re-routes dynamic edges, settles skips, and refills the freed slot.
Scheduling decisions are serialized through a single pump chain, so
completions can never double-dispatch a node.

Slot scheduling makes a new interleaving possible that barrier batching
never saw: a rework route or conditional route can target a node that is
still executing. Those activations are deferred and applied when the
target lands (with the rework budget re-checked at activation time, so
two failures deferring onto one fixer cannot jointly overspend it).

Phase ordering inside each pump pass is preserved exactly from the old
loop — route evaluation, then the approval pause (skips do not settle
past a pause), then skip propagation, then dispatch — so persisted-state
semantics are unchanged; all 8 existing graph-execution tests pass
unmodified.

Benchmark (concurrency 2, one 3s straggler beside a 100ms -> 1s -> 1s
chain, 2 runs each):
  barrier: 4244 / 4236 ms
  slots:    3048 / 3053 ms   (-28%)

A regression test pins the scheduling property: a node depending only
on a quick sibling starts while the straggler is still in flight.
@momozi1996
momozi1996 merged commit 49c95bb into momozi1996:main Oct 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants