Repository navigation
perf(graph): work-conserving slot scheduler replaces barrier batching - #12
Merged
Merged
Conversation
executeGraph ran ready nodes in barrier batches: Promise.all over each batch, so the slowest node of every batch gated the dispatch of the next. LLM subagents have wildly heterogeneous durations (retries can triple a node's wall time), so fast nodes finished and left slots idle while dependents of quick parents waited behind unrelated stragglers. Makespan degenerated into the sum of per-batch maxima instead of tracking the concurrency-limited critical path. Replace the batch loop with a work-conserving scheduler: up to concurrency nodes stay in flight and every completion immediately re-routes dynamic edges, settles skips, and refills the freed slot. Scheduling decisions are serialized through a single pump chain, so completions can never double-dispatch a node. Slot scheduling makes a new interleaving possible that barrier batching never saw: a rework route or conditional route can target a node that is still executing. Those activations are deferred and applied when the target lands (with the rework budget re-checked at activation time, so two failures deferring onto one fixer cannot jointly overspend it). Phase ordering inside each pump pass is preserved exactly from the old loop — route evaluation, then the approval pause (skips do not settle past a pause), then skip propagation, then dispatch — so persisted-state semantics are unchanged; all 8 existing graph-execution tests pass unmodified. Benchmark (concurrency 2, one 3s straggler beside a 100ms -> 1s -> 1s chain, 2 runs each): barrier: 4244 / 4236 ms slots: 3048 / 3053 ms (-28%) A regression test pins the scheduling property: a node depending only on a quick sibling starts while the straggler is still in flight.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
executeGraphscheduled ready nodes in barrier batches:readyNodes().slice(0, concurrency)followed byawait Promise.all(batch). The slowest node of every batch gated the dispatch of the next batch. LLM subagent durations are wildly heterogeneous (a node withmaxRetries=2can legitimately run 3× a sibling's wall time), so:Makespan degenerated into Σ per-batch maxima instead of tracking the concurrency-limited critical path. Worst case: level 1 =
[5s node, 300s node], a node depending only on the 5s one still waits 300s.Change
Replace the batch loop with a work-conserving scheduler:
concurrencynodes stay in flight; every completion immediately re-routes dynamic edges, settles skips, and refills the freed slot.pump()chain (completions enqueue onto it), so concurrent completions can never double-dispatch a node.deferredActivations) and applied when the target lands; the rework budget is re-checked at activation time so two failures deferring onto the same fixer cannot jointly overspend it.onNodeordering all behave exactly as before — all 8 existing graph-execution tests pass unmodified.Benchmark
Same DAG under both schedulers (
concurrency: 2): one 3s straggler independent of a100ms → 1s → 1schain.−28% makespan on a 4-node graph; the gain grows with graph width and duration variance (real LLM graphs vary 10s–300s per node). Reproduce: fixture child that sleeps
N msviaAtomics.wait, driven through the realspawnSubagentpath withprocess.argv[1]pointed at it (same technique as the existing tests).Regression test
New test pins the scheduling property, not a timing constant: a node depending only on a quick sibling must start while the straggler is still in flight (
startedAt(after) + 800 < finishedAt(slow)), with the dependency edge still respected causally. The 800 ms margin keeps it robust on loaded CI runners.Verification
npx tsx --test src/test/*.test.ts),tsc --noEmitcleangraph-executiontests pass unmodified (approval pauses, rework loops, skip propagation, resume, budget rail)