Skip to content

research(orchestration): VMAO adaptive replanning — LLM-judged completeness triggers gap-filling DAG replay (arXiv:2603.11445) #2252

@bug-ops

Description

@bug-ops

Source

arXiv:2603.11445 — Verified Multi-Agent Orchestration: A Plan-Execute-Verify-Replan Framework (Mar 2026)
https://arxiv.org/abs/2603.11445

Summary

VMAO decomposes queries into dependency DAGs, executes sub-tasks in parallel with context propagation, then runs an LLM verifier that produces a completeness score + gap list. Gaps trigger targeted replanning (only the incomplete sub-tasks are re-queued, not the full DAG). On 25 expert market-research queries: answer completeness improved 3.1→4.2, source quality 2.6→4.1 (1–5 scale) vs. static multi-agent baselines.

Relevance to Zeph

Zeph already has DagScheduler, LlmPlanner, and the PEVR (PlanVerifier) skeleton from PR #2235, but verify_completeness = false by default. The missing piece is:

  1. A verifier LLM call after DAG execution that produces structured completeness assessment
  2. Selective re-queuing of failed/incomplete sub-tasks using existing DagScheduler infrastructure

VMAO's pattern slots directly onto OrchestrationConfig.planner_provider — the verifier uses a verifier_provider field (same pool reference pattern).

Implementation Sketch

  • Add verifier_provider field to [orchestration] config (empty = skip verification)
  • After DagScheduler completes, call verifier with: plan + aggregated sub-agent outputs
  • Verifier returns: completeness score + list of gap sub-tasks
  • If score < threshold: re-queue gap sub-tasks as a new partial DAG (max 1 replan cycle to avoid loops)
  • verify_completeness = true in OrchestrationConfig activates this path

Complexity

LOW–MEDIUM — verification step is a single LLM call with structured output; replanning reuses existing LlmPlanner. No new data structures needed.

Component

zeph-core orchestration (DagScheduler, LlmPlanner), OrchestrationConfig — adds verifier_provider field

Metadata

Metadata

Assignees

Labels

P2High value, medium complexityllmzeph-llm crate (Ollama, Claude)researchResearch-driven improvement

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions