Skip to content

research(orchestration): async parallel task dispatch in DagScheduler (DynTaskMAS pattern) #1628

@bug-ops

Description

@bug-ops

Research Finding

DynTaskMAS (arXiv:2503.07675, Mar 2026) reports 21-33% execution time reduction and near-linear throughput scaling to 16 concurrent agents using an Asynchronous Parallel Execution Engine. Instead of tick-based sequential dispatch, tasks ready after dependency resolution are dispatched concurrently via async join/join_all.

Applicability

Zeph's DagScheduler dispatches ready tasks sequentially per tick. If N tasks become ready simultaneously (e.g., after a shared dependency completes), they are dispatched one per tick with 250ms deferral between them. With async parallel dispatch:

  • All ready tasks at the end of a tick would be spawned concurrently
  • Execution time for independent parallel branches would approach single-task time

Current Constraint

SubAgentManager.max_concurrent still limits parallelism at the execution layer. This is correct behavior — the change would be at the scheduler dispatch level, not the concurrency limit.

Precondition

PERF-SC-04 (cron expression parsing bug) should be fixed first. See also #1619 (planning sub-agent starves execution slots).

Design Sketch

In DagScheduler::tick():

  1. Collect all ready tasks (not just first one)
  2. Attempt spawn_for_task for each in a single async batch
  3. Return all spawned + deferred results
  4. Update exponential backoff only for the deferred set

Source

Research session 2026-03-13. arXiv:2503.07675 (DynTaskMAS).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestresearchResearch-driven improvement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions