Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Agentic-R: Learning to Retrieve for Agentic Search

Wenhan Liu, Xinyu Ma, Yutao Zhu, Yuchen Li, Daiting Shi, Dawei Yin, Zhicheng Dou

TL;DR

This work tackles the retriever design gap in agentic search by proposing Agentic-R, a retriever trained to optimize passage utility across multi-turn reasoning. It models passage utility with two signals: local query-passage relevance and global final-answer correctness, using LLM-based listwise scoring and final EM-based evaluation. An iterative agent-retriever optimization framework trains the agent and retriever in a bidirectional loop, yielding higher-quality agent trajectories and progressively stronger retrieval signals. Empirical results across seven QA benchmarks show that Agentic-R consistently boosts performance, reduces the number of search turns, and generalizes across diverse search agents. This approach advances efficient, multi-turn retrieval-augmented reasoning for complex question answering tasks.

Abstract

Agentic search has recently emerged as a powerful paradigm, where an agent interleaves multi-step reasoning with on-demand retrieval to solve complex questions. Despite its success, how to design a retriever for agentic search remains largely underexplored. Existing search agents typically rely on similarity-based retrievers, while similar passages are not always useful for final answer generation. In this paper, we propose a novel retriever training framework tailored for agentic search. Unlike retrievers designed for single-turn retrieval-augmented generation (RAG) that only rely on local passage utility, we propose to use both local query-passage relevance and global answer correctness to measure passage utility in a multi-turn agentic search. We further introduce an iterative training strategy, where the search agent and the retriever are optimized bidirectionally and iteratively. Different from RAG retrievers that are only trained once with fixed questions, our retriever is continuously improved using evolving and higher-quality queries from the agent. Extensive experiments on seven single-hop and multi-hop QA benchmarks demonstrate that our retriever, termed \ours{}, consistently outperforms strong baselines across different search agents. Our codes are available at: https://github.com/8421BCD/Agentic-R.

Agentic-R: Learning to Retrieve for Agentic Search

TL;DR

This work tackles the retriever design gap in agentic search by proposing Agentic-R, a retriever trained to optimize passage utility across multi-turn reasoning. It models passage utility with two signals: local query-passage relevance and global final-answer correctness, using LLM-based listwise scoring and final EM-based evaluation. An iterative agent-retriever optimization framework trains the agent and retriever in a bidirectional loop, yielding higher-quality agent trajectories and progressively stronger retrieval signals. Empirical results across seven QA benchmarks show that Agentic-R consistently boosts performance, reduces the number of search turns, and generalizes across diverse search agents. This approach advances efficient, multi-turn retrieval-augmented reasoning for complex question answering tasks.

Abstract

Agentic search has recently emerged as a powerful paradigm, where an agent interleaves multi-step reasoning with on-demand retrieval to solve complex questions. Despite its success, how to design a retriever for agentic search remains largely underexplored. Existing search agents typically rely on similarity-based retrievers, while similar passages are not always useful for final answer generation. In this paper, we propose a novel retriever training framework tailored for agentic search. Unlike retrievers designed for single-turn retrieval-augmented generation (RAG) that only rely on local passage utility, we propose to use both local query-passage relevance and global answer correctness to measure passage utility in a multi-turn agentic search. We further introduce an iterative training strategy, where the search agent and the retriever are optimized bidirectionally and iteratively. Different from RAG retrievers that are only trained once with fixed questions, our retriever is continuously improved using evolving and higher-quality queries from the agent. Extensive experiments on seven single-hop and multi-hop QA benchmarks demonstrate that our retriever, termed \ours{}, consistently outperforms strong baselines across different search agents. Our codes are available at: https://github.com/8421BCD/Agentic-R.
Paper Structure (41 sections, 6 equations, 8 figures, 6 tables, 1 algorithm)

This paper contains 41 sections, 6 equations, 8 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Agentic Search
  4. Training Retrievers for Generation
  5. Preliminary: Agentic Search
  6. Methodology
  7. Agentic-R
  8. Training Data Construction
  9. Local Relevance.
  10. Final Answer Correctness.
  11. Positive and Negative Passages Selection.
  12. Training Approach
  13. Training Input.
  14. Training Loss.
  15. Agent-Retriever Iterative Optimization
  16. ...and 26 more sections

Figures (8)

  • Figure 1: An example of agentic search.
  • Figure 2: An overview of our training framework.
  • Figure 3: The comparison of search turns on different retrievers.
  • Figure 4: Effect of different iteration numbers ($K$) in agent--retriever optimization. Different colors on the x-axis indicate different iteration number.
  • Figure 5: The input prompt for training our search agent.
  • ...and 3 more figures