Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Verified Multi-Agent Orchestration: A Plan-Execute-Verify-Replan Framework for Complex Query Resolution

Xing Zhang, Yanwei Cui, Guanghui Wang, Qucy Wei Qiu, Ziyuan Li, Fangwei Han, Yajing Huang, Hengzhi Qiu, Bin Zhu, Peiyang He

TL;DR

Verified Multi-Agent Orchestration is presented, a framework that coordinates specialized LLM-based agents through a verification-driven iterative loop and improves answer completeness and source quality compared to a single-agent baseline, demonstrating that orchestration-level verification is an effective mechanism for multi-agent quality assurance.

Abstract

We present Verified Multi-Agent Orchestration (VMAO), a framework that coordinates specialized LLM-based agents through a verification-driven iterative loop. Given a complex query, our system decomposes it into a directed acyclic graph (DAG) of sub-questions, executes them through domain-specific agents in parallel, verifies result completeness via LLM-based evaluation, and adaptively replans to address gaps. The key contributions are: (1) dependency-aware parallel execution over a DAG of sub-questions with automatic context propagation, (2) verification-driven adaptive replanning that uses an LLM-based verifier as an orchestration-level coordination signal, and (3) configurable stop conditions that balance answer quality against resource usage. On 25 expert-curated market research queries, VMAO improves answer completeness from 3.1 to 4.2 and source quality from 2.6 to 4.1 (1-5 scale) compared to a single-agent baseline, demonstrating that orchestration-level verification is an effective mechanism for multi-agent quality assurance.

Verified Multi-Agent Orchestration: A Plan-Execute-Verify-Replan Framework for Complex Query Resolution

TL;DR

Verified Multi-Agent Orchestration is presented, a framework that coordinates specialized LLM-based agents through a verification-driven iterative loop and improves answer completeness and source quality compared to a single-agent baseline, demonstrating that orchestration-level verification is an effective mechanism for multi-agent quality assurance.

Abstract

We present Verified Multi-Agent Orchestration (VMAO), a framework that coordinates specialized LLM-based agents through a verification-driven iterative loop. Given a complex query, our system decomposes it into a directed acyclic graph (DAG) of sub-questions, executes them through domain-specific agents in parallel, verifies result completeness via LLM-based evaluation, and adaptively replans to address gaps. The key contributions are: (1) dependency-aware parallel execution over a DAG of sub-questions with automatic context propagation, (2) verification-driven adaptive replanning that uses an LLM-based verifier as an orchestration-level coordination signal, and (3) configurable stop conditions that balance answer quality against resource usage. On 25 expert-curated market research queries, VMAO improves answer completeness from 3.1 to 4.2 and source quality from 2.6 to 4.1 (1-5 scale) compared to a single-agent baseline, demonstrating that orchestration-level verification is an effective mechanism for multi-agent quality assurance.
Paper Structure (15 sections, 3 figures, 5 tables, 1 algorithm)

This paper contains 15 sections, 3 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Framework Architecture
  4. Overview
  5. Planning and Execution
  6. Verification, Replanning, and Synthesis
  7. Implementation
  8. Experiments
  9. Dataset: Market Research Queries
  10. Baselines and Configuration
  11. Results
  12. Discussion
  13. Conclusion
  14. Prompt Templates
  15. Configuration Parameters

Figures (3)

  • Figure 1: (a) VMAO framework architecture showing the iterative Plan-Execute-Verify-Replan loop. (b) Agent taxonomy organized by functional tier with information flow from data gathering through analysis to output generation.
  • Figure 2: (a) DAG execution: independent sub-questions execute in Wave 1; dependent questions in subsequent waves. (b) Verification-driven iteration: Iteration 1 identifies incomplete results, triggering replanning; Iteration 2 achieves sufficient completeness for synthesis.
  • Figure 3: (a) Token usage breakdown by orchestration phase for a typical query. Execution dominates at 61%, while verification and synthesis remain efficient. (b) Completeness scores by query category across methods. VMAO shows consistent improvements, with largest gains on Strategic Assessment (+53%).