Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FREESON: Retriever-Free Retrieval-Augmented Reasoning via Corpus-Traversing MCTS

Chaeeun Kim, Seungone Kim

TL;DR

FREESON tackles the bottleneck of external retrievers in retrieval-augmented reasoning by enabling a single large reasoning model to act as both generator and retriever. It introduces Corpus-Traversing Monte Carlo Tree Search (CT-MCTS), a token-level, prefix-constrained search that traverses a corpus via a dynamically built CorpusTree, with multi-token node expansions and an on-policy value network to guide retrieval toward answer-containing regions. Through on-policy CT-MCTS rollouts and synthetic-data–augmented training, FREESON achieves an average EM/F1 improvement of 14.4% over multi-step baselines with separate retrievers across five open-domain QA benchmarks, and outperforms many baselines, sometimes surpassing the strongest by a few percentage points. The approach eliminates the need for external search engines or retriever hardware, offering a retriever-free pathway for knowledge-intensive reasoning and a foundation for domain-specific applications with unlabeled corpora. Limitations include dependence on a predefined corpus and potential extensions using reinforcement learning to further optimize traversal strategies.

Abstract

Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in multi-step reasoning and calling search engines at appropriate steps. However, existing retrieval-augmented reasoning approaches rely on separate retrieval models, limiting the LRM's role in retrieval to deciding when to retrieve and how to query. This separation not only increases hardware and operational costs but also leads to errors in the retrieval process due to the representation bottleneck, a phenomenon where the retriever's embedding space is not expressive enough to meet the generator's requirements. To address this, we shift our perspective from sequence-to-sequence matching to locating the answer-containing paths within the corpus, and propose a novel framework called FREESON (Retriever-FREE Retrieval-Augmented ReaSONing). This framework enables LRMs to retrieve relevant knowledge on their own by acting as both a generator and retriever. To achieve this, we introduce a variant of the MCTS algorithm specialized for the retrieval task, which we call CT-MCTS (Corpus-Traversing Monte Carlo Tree Search). In this algorithm, LRMs traverse through the corpus toward answer-containing regions. Our results on five open-domain QA benchmarks, including single-hop and multi-hop questions, show that FREESON achieves an average improvement of 14.4% in EM and F1 over four multi-step reasoning models with a separate retriever, and it also performs comparably to the strongest baseline, surpassing it by 3% on PopQA and 2WikiMultihopQA.

FREESON: Retriever-Free Retrieval-Augmented Reasoning via Corpus-Traversing MCTS

TL;DR

FREESON tackles the bottleneck of external retrievers in retrieval-augmented reasoning by enabling a single large reasoning model to act as both generator and retriever. It introduces Corpus-Traversing Monte Carlo Tree Search (CT-MCTS), a token-level, prefix-constrained search that traverses a corpus via a dynamically built CorpusTree, with multi-token node expansions and an on-policy value network to guide retrieval toward answer-containing regions. Through on-policy CT-MCTS rollouts and synthetic-data–augmented training, FREESON achieves an average EM/F1 improvement of 14.4% over multi-step baselines with separate retrievers across five open-domain QA benchmarks, and outperforms many baselines, sometimes surpassing the strongest by a few percentage points. The approach eliminates the need for external search engines or retriever hardware, offering a retriever-free pathway for knowledge-intensive reasoning and a foundation for domain-specific applications with unlabeled corpora. Limitations include dependence on a predefined corpus and potential extensions using reinforcement learning to further optimize traversal strategies.

Abstract

Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in multi-step reasoning and calling search engines at appropriate steps. However, existing retrieval-augmented reasoning approaches rely on separate retrieval models, limiting the LRM's role in retrieval to deciding when to retrieve and how to query. This separation not only increases hardware and operational costs but also leads to errors in the retrieval process due to the representation bottleneck, a phenomenon where the retriever's embedding space is not expressive enough to meet the generator's requirements. To address this, we shift our perspective from sequence-to-sequence matching to locating the answer-containing paths within the corpus, and propose a novel framework called FREESON (Retriever-FREE Retrieval-Augmented ReaSONing). This framework enables LRMs to retrieve relevant knowledge on their own by acting as both a generator and retriever. To achieve this, we introduce a variant of the MCTS algorithm specialized for the retrieval task, which we call CT-MCTS (Corpus-Traversing Monte Carlo Tree Search). In this algorithm, LRMs traverse through the corpus toward answer-containing regions. Our results on five open-domain QA benchmarks, including single-hop and multi-hop questions, show that FREESON achieves an average improvement of 14.4% in EM and F1 over four multi-step reasoning models with a separate retriever, and it also performs comparably to the strongest baseline, surpassing it by 3% on PopQA and 2WikiMultihopQA.

Paper Structure

This paper contains 36 sections, 2 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dynamically Adapted Search Space
  4. CT-MCTS for Autonomous Retrieval
  5. Selection.
  6. Granularity-aware multi-node expansion.
  7. Evaluating answer presence in search trajectories.
  8. Training the Value Network
  9. On-policy training on CT-MCTS rollouts.
  10. Off-policy training on synthetic trajectories.
  11. Document Selection From Retrieved Paths
  12. Experiments
  13. Benchmarks
  14. Baselines
  15. Main Results
  16. ...and 21 more sections

Figures (2)

  • Figure 1: Overview of retrieval-augmented reasoning process. Left:FREESON performs both reasoning and retrieval using a single generator model via token-level CT-MCTS. Right: conventional RAR methods compute similarity scores between query and document embeddings using a separate retrieval model. FREESON fully leverages the LRM’s reasoning over document structure, enabling precise access to relevant information from the corpus at inference time without relying on memorization.
  • Figure 2: Impact of node granularity on system efficiency and performance (1 -- 10 tokens per node). Left: Latency (s) sharply decreases with higher granularity, enabling faster search. Middle: As granularity increases, the retrieved path becomes longer (more informative), while the rollout count remains similar—indicating more efficient search per computation. Right: Higher granularity leads to better performance.