Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search

Yiqun Chen, Lingyong Yan, Zixuan Yang, Erhan Zhang, Jiashu Zhao, Shuaiqiang Wang, Dawei Yin, Jiaxin Mao

TL;DR

The paper tackles instability in agentic search by separating planning and information integration into two specialized agent groups: Search Behavior Agents and Knowledge Management Agents. It introduces turn-level dense rewards and a parameter-shared PPO framework to jointly optimize all roles, yielding improved accuracy and training stability on complex multi-hop QA tasks. Empirical results across seven benchmarks show M-ASK outperforming monolithic and modular baselines, with strong generalization to unseen domains and notably better convergence behavior. The work demonstrates that explicit role specialization and intermediate supervision are key to scaling agentic search under noisy real-world conditions.

Abstract

Agentic search has emerged as a promising paradigm for complex information seeking by enabling Large Language Models (LLMs) to interleave reasoning with tool use. However, prevailing systems rely on monolithic agents that suffer from structural bottlenecks, including unconstrained reasoning outputs that inflate trajectories, sparse outcome-level rewards that complicate credit assignment, and stochastic search noise that destabilizes learning. To address these challenges, we propose \textbf{M-ASK} (Multi-Agent Search and Knowledge), a framework that explicitly decouples agentic search into two complementary roles: Search Behavior Agents, which plan and execute search actions, and Knowledge Management Agents, which aggregate, filter, and maintain a compact internal context. This decomposition allows each agent to focus on a well-defined subtask and reduces interference between search and context construction. Furthermore, to enable stable coordination, M-ASK employs turn-level rewards to provide granular supervision for both search decisions and knowledge updates. Experiments on multi-hop QA benchmarks demonstrate that M-ASK outperforms strong baselines, achieving not only superior answer accuracy but also significantly more stable training dynamics.\footnote{The source code for M-ASK is available at https://github.com/chenyiqun/M-ASK.}

Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search

TL;DR

The paper tackles instability in agentic search by separating planning and information integration into two specialized agent groups: Search Behavior Agents and Knowledge Management Agents. It introduces turn-level dense rewards and a parameter-shared PPO framework to jointly optimize all roles, yielding improved accuracy and training stability on complex multi-hop QA tasks. Empirical results across seven benchmarks show M-ASK outperforming monolithic and modular baselines, with strong generalization to unseen domains and notably better convergence behavior. The work demonstrates that explicit role specialization and intermediate supervision are key to scaling agentic search under noisy real-world conditions.

Abstract

Agentic search has emerged as a promising paradigm for complex information seeking by enabling Large Language Models (LLMs) to interleave reasoning with tool use. However, prevailing systems rely on monolithic agents that suffer from structural bottlenecks, including unconstrained reasoning outputs that inflate trajectories, sparse outcome-level rewards that complicate credit assignment, and stochastic search noise that destabilizes learning. To address these challenges, we propose \textbf{M-ASK} (Multi-Agent Search and Knowledge), a framework that explicitly decouples agentic search into two complementary roles: Search Behavior Agents, which plan and execute search actions, and Knowledge Management Agents, which aggregate, filter, and maintain a compact internal context. This decomposition allows each agent to focus on a well-defined subtask and reduces interference between search and context construction. Furthermore, to enable stable coordination, M-ASK employs turn-level rewards to provide granular supervision for both search decisions and knowledge updates. Experiments on multi-hop QA benchmarks demonstrate that M-ASK outperforms strong baselines, achieving not only superior answer accuracy but also significantly more stable training dynamics.\footnote{The source code for M-ASK is available at https://github.com/chenyiqun/M-ASK.}
Paper Structure (60 sections, 5 equations, 6 figures, 6 tables)

This paper contains 60 sections, 5 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. From Iterative RAG to Agentic Search
  4. Context Management and Optimization
  5. Multi-Agent Systems for Information Retrieval
  6. Method
  7. Problem Formulation
  8. Structured Knowledge State
  9. M-ASK Architecture
  10. Search Behavior Agents (SBA)
  11. Planning Agent ($A_{\text{plan}}$)
  12. Search Agent ($A_{\text{search}}$)
  13. Answer Agent ($A_{\text{ans}}$)
  14. Knowledge Management Agents (KMA)
  15. Summary Agent ($A_{\text{sum}}$)
  16. ...and 45 more sections

Figures (6)

  • Figure 1: Challenges of current monolithic methods and the M-ASK solution. Existing agents struggle with the long-horizon credit assignment problem caused by unconstrained output length, sparse rewards and search noise. M-ASK addresses these bottlenecks through role decoupling and turn-level dense rewards.
  • Figure 2: Overview of the M-ASK framework. (1) Rollout: The Planning Agent initializes the state $\mathcal{K}_0$, followed by an iterative loop where Search and Knowledge Management Agents refine the trajectory. Crucially, the Answer Agent updates the prediction after each turn. (2) Training: A hybrid reward mechanism assigns absolute scores ($F_1^0$ and $F_1^t$) to the Planning and Answer Agents, respectively, while the collaborative agents (Search, Summary, Update) share the marginal improvement ($\Delta F_1^t$) to incentivize step-wise refinement.
  • Figure : (a) Search-r1 Training Dynamics
  • Figure : (a) Search-r1 Training Dynamics
  • Figure : (b) Average Response Length
  • ...and 1 more figures