A-MapReduce: Executing Wide Search via Agentic MapReduce

TL;DR

A-MapReduce reframes wide-search tasks for LLM-based multi-agent systems as a horizontally structured, MapReduce-like computation, enabling explicit coverage tracking, parallelized retrieval, and schema-consistent aggregation. It introduces an experiential memory that distills past execution trajectories into query-conditioned priors, guiding adaptive task decomposition, batching, and template decisions to improve efficiency and robustness. Across five benchmarks, the approach achieves state-of-the-art performance and favorable cost–performance trade-offs, while ablations and case studies demonstrate the pivotal role of memory in accelerating convergence and reducing redundant work. The work offers a practical framework for scalable, open-world wide search with potential impact on research tooling, evidence synthesis, and decision support, complemented by an open-source implementation.

Abstract

Contemporary large language model (LLM)-based multi-agent systems exhibit systematic advantages in deep research tasks, which emphasize iterative, vertically structured information seeking. However, when confronted with wide search tasks characterized by large-scale, breadth-oriented retrieval, existing agentic frameworks, primarily designed around sequential, vertically structured reasoning, remain stuck in expansive search objectives and inefficient long-horizon execution. To bridge this gap, we propose A-MapReduce, a MapReduce paradigm-inspired multi-agent execution framework that recasts wide search as a horizontally structured retrieval problem. Concretely, A-MapReduce implements parallel processing of massive retrieval targets through task-adaptive decomposition and structured result aggregation. Meanwhile, it leverages experiential memory to drive the continual evolution of query-conditioned task allocation and recomposition, enabling progressive improvement in large-scale wide-search regimes. Extensive experiments on five agentic benchmarks demonstrate that A-MapReduce is (i) high-performing, achieving state-of-the-art performance on WideSearch and DeepWideSearch, and delivering 5.11% - 17.50% average Item F1 improvements compared with strong baselines with OpenAI o3 or Gemini 2.5 Pro backbones; (ii) cost-effective and efficient, delivering superior cost-performance trade-offs and reducing running time by 45.8\% compared to representative multi-agent baselines. The code is available at https://github.com/mingju-c/AMapReduce.

Figures (9)

• Figure 1: Structural isomorphism: A-MapReduce mirrors the MapReduce pipeline via a one-to-one operator corresponding.
• Figure 2: Overall workflow of our A-MapReduce. Given a wide-search query, our framework takes lightweight sequential execution, then retrieves experiential prior as a decision anchor from experiential memory and samples a conditional MapReduce decision. It then decomposes the query into atomic tasks for batched parallel execution and reduces partial results into a single structured output. After completion, environmental feedback updates the experiential memory, enabling continual cross-task refinement of execution decisions.
• Figure 3: Cost--performance trade-offs of A-MapReduce across benchmarks.
• Figure 4: Sensitivity analysis on WideSearch: (a) robustness across backbones; (b) sensitivity to the number of retrieved insights.
• Figure 5: Case study on a representative WideSearch query. Left: a general MAS executes year-by-year sequential retrieval, where long-horizon context accumulation can break structured-table construction. Right:A-MapReduce instantiates the query as a schema-driven task matrix and runs MapReduce-style execution, shown for variants without and with experiential memory.