-
Notifications
You must be signed in to change notification settings - Fork 2
research(orchestration): async parallel task dispatch in DagScheduler (DynTaskMAS pattern) #1628
Description
Research Finding
DynTaskMAS (arXiv:2503.07675, Mar 2026) reports 21-33% execution time reduction and near-linear throughput scaling to 16 concurrent agents using an Asynchronous Parallel Execution Engine. Instead of tick-based sequential dispatch, tasks ready after dependency resolution are dispatched concurrently via async join/join_all.
Applicability
Zeph's DagScheduler dispatches ready tasks sequentially per tick. If N tasks become ready simultaneously (e.g., after a shared dependency completes), they are dispatched one per tick with 250ms deferral between them. With async parallel dispatch:
- All ready tasks at the end of a tick would be spawned concurrently
- Execution time for independent parallel branches would approach single-task time
Current Constraint
SubAgentManager.max_concurrent still limits parallelism at the execution layer. This is correct behavior — the change would be at the scheduler dispatch level, not the concurrency limit.
Precondition
PERF-SC-04 (cron expression parsing bug) should be fixed first. See also #1619 (planning sub-agent starves execution slots).
Design Sketch
In DagScheduler::tick():
- Collect all ready tasks (not just first one)
- Attempt
spawn_for_taskfor each in a single async batch - Return all spawned + deferred results
- Update exponential backoff only for the deferred set
Source
Research session 2026-03-13. arXiv:2503.07675 (DynTaskMAS).