DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking

TL;DR

DeepWideSearch tackles a critical gap in evaluating information-seeking agents by jointly examining depth (multi-hop reasoning) and width (broad information collection). It introduces two data-conversion pipelines, Deep2Wide and Wide2Deep, to generate 220 questions across 15 domains, and establishes a three-fold evaluation framework spanning depth, width, and efficiency. Across diverse baselines, results reveal a substantial challenge: average success remains around 2.39%, with agents improving core entity identification but often failing to complete comprehensive table-level retrieval due to issues like lack of reflection and context overflow. Four failure modes are identified (lack of reflection, reliance on internal knowledge, insufficient retrieval, context overflow), informing future architectural directions for robust deep-wide search. The authors publicly release both the DeepWideSearch benchmark and evaluation code to catalyze progress toward more capable and robust information-seeking agents.

Abstract

Current search agents fundamentally lack the ability to simultaneously perform \textit{deep} reasoning over multi-hop retrieval and \textit{wide}-scale information collection-a critical deficiency for real-world applications like comprehensive market analysis and business development. To bridge this gap, we introduce DeepWideSearch, the first benchmark explicitly designed to evaluate agents to integrate depth and width in information seeking. In DeepWideSearch, agents must process a large volume of data, each requiring deep reasoning over multi-hop retrieval paths. Specifically, we propose two methods to converse established datasets, resulting in a curated collection of 220 questions spanning 15 diverse domains. Extensive experiments demonstrate that even state-of-the-art agents achieve only 2.39% average success rate on DeepWideSearch, highlighting the substantial challenge of integrating depth and width search in information-seeking tasks. Furthermore, our error analysis reveals four failure modes: lack of reflection, overreliance on internal knowledge, insufficient retrieval, and context overflow-exposing key limitations in current agent architectures. We publicly release DeepWideSearch to catalyze future research on more capable and robust information-seeking agents.

Figures (17)

• Figure 1: The comparison of existing benchmarks on search width and depth.
• Figure 2: The detailed comparison among deep search, wide search benchmarks and our proposed DeepWideSearch.
• Figure 3: Task formulation of DeepWideSearch task. The evaluation metrics (highlighted in red) are detailed in Section \ref{['subsec.evaluation_metric']}.
• Figure 4: The pipelines of our proposed Deep2Wide and Wide2Deep data construction methods.
• Figure 5: One deep search question in BrowseComp-ZH.