Holistically Guided Monte Carlo Tree Search for Intricate Information Seeking

TL;DR

HG-MCTS introduces a holistic information-seeking paradigm that combines an adaptive checklist with multi-perspective reward modeling to guide Monte Carlo Tree Search for multi-step web retrieval. By maintaining a knowledge memory and updating a dynamic sub-goal checklist, the method balances thorough information collection with targeted exploration, reducing redundancy and omissions. Empirical results across five challenging multi-hop QA benchmarks show consistent performance gains and robustness across backbone LLMs, underscoring the approach's practicality for real-world, information-intensive tasks. The framework advances interpretable, flexible retrieval systems by integrating global guidance, explicit sub-goals, and qualitative feedback into the search process.

Abstract

In the era of vast digital information, the sheer volume and heterogeneity of available information present significant challenges for intricate information seeking. Users frequently face multistep web search tasks that involve navigating vast and varied data sources. This complexity demands every step remains comprehensive, accurate, and relevant. However, traditional search methods often struggle to balance the need for localized precision with the broader context required for holistic understanding, leaving critical facets of intricate queries underexplored. In this paper, we introduce an LLM-based search assistant that adopts a new information seeking paradigm with holistically guided Monte Carlo tree search (HG-MCTS). We reformulate the task as a progressive information collection process with a knowledge memory and unite an adaptive checklist with multi-perspective reward modeling in MCTS. The adaptive checklist provides explicit sub-goals to guide the MCTS process toward comprehensive coverage of complex user queries. Simultaneously, our multi-perspective reward modeling offers both exploration and retrieval rewards, along with progress feedback that tracks completed and remaining sub-goals, refining the checklist as the tree search progresses. By striking a balance between localized tree expansion and global guidance, HG-MCTS reduces redundancy in search paths and ensures that all crucial aspects of an intricate query are properly addressed. Extensive experiments on real-world intricate information seeking tasks demonstrate that HG-MCTS acquires thorough knowledge collections and delivers more accurate final responses compared with existing baselines.

Figures (5)

• Figure 1: Illustration of the pitfalls in handling intricate queries. Typical reasoning methods with web search often collect non-comprehensive documents (left), while HG-MCTS can effectively capture all necessary documents (right).
• Figure 2: The overall framework of the proposed HG-MCTS method. The left panel outlines the iterative Monte Carlo tree search procedure with a global checklist and memory mechanism, including different actions in MCTS. The right panel provides a detailed explanation of node expansion and the corresponding reward modeling process with quantitative progress reward and progress feedback.
• Figure 3: An illustration of the checklist corresponding to an intricate query encompassing multiple sub-goals.
• Figure 4: Information collection evaluation for different methods on Recall rate (blue part).
• Figure 5: Evaluation results of HG-MCTS with various simulation numbers employed by different LLMs on FanoutQA.