Talk to Right Specialists: Routing and Planning in Multi-agent System for Question Answering

TL;DR

RopMura proposes a router and planner to coordinate multiple RAG-based agents, each owning domain-specific knowledge, for cross-domain question answering while preserving knowledge sovereignty. The router uses embedding-based centroids to define knowledge boundaries and select the most relevant agents; the planner recursively decomposes multi-hop questions into steps that the router can execute across agents. Experiments on Natural Questions, HotpotQA, and Multi-hop RAG show that routing alone improves single-hop precision, and the combined routing-planning approach achieves accurate multi-hop answers, at the cost of higher token usage. The work offers a scalable approach to cross-domain QA with data governance considerations.

Abstract

Leveraging large language models (LLMs), an agent can utilize retrieval-augmented generation (RAG) techniques to integrate external knowledge and increase the reliability of its responses. Current RAG-based agents integrate single, domain-specific knowledge sources, limiting their ability and leading to hallucinated or inaccurate responses when addressing cross-domain queries. Integrating multiple knowledge bases into a unified RAG-based agent raises significant challenges, including increased retrieval overhead and data sovereignty when sensitive data is involved. In this work, we propose RopMura, a novel multi-agent system that addresses these limitations by incorporating highly efficient routing and planning mechanisms. RopMura features two key components: a router that intelligently selects the most relevant agents based on knowledge boundaries and a planner that decomposes complex multi-hop queries into manageable steps, allowing for coordinating cross-domain responses. Experimental results demonstrate that RopMura effectively handles both single-hop and multi-hop queries, with the routing mechanism enabling precise answers for single-hop queries and the combined routing and planning mechanisms achieving accurate, multi-step resolutions for complex queries.

Figures (8)

• Figure 1: Routing mechanism under Single-hop QA with an example
• Figure 2: Workflow of $\text{RopMura}$ under Multi-hop QA with a three-round example
• Figure 3: Prompt Templates for the single-hop QA task (i.e., Natural Questions).
• Figure 4: Prompt Templates for the Multihop QA tasks (i.e., HotpotQA and Multi-Hop RAG).
• Figure 5: Prompt Templates for the Multihop QA tasks (i.e., HotpotQA and Multi-Hop RAG) (cont.)