SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback

TL;DR

SAGE introduces a steerable two-agent framework that automatically generates high-quality, difficulty-controlled deep-search QA data by leveraging execution feedback from a solving search agent. By coupling a data generator with iterative feedback from the search agent, the approach improves the correctness and complexity of synthetic QAs and yields strong downstream gains when used to train deep search agents, including substantial gains on in-domain tasks and transfer abilities to Google Search at inference. Intrinsic analysis shows the generated data spans diverse reasoning strategies, while extrinsic evaluation demonstrates up to 27-29% relative improvements on multiple benchmarks across model sizes and domains. The work highlights the potential of execution-based data refinement to scale high-quality multi-hop QA data without costly human annotation, with practical implications for robust deep search systems.

Abstract

Deep search agents, which aim to answer complex questions requiring reasoning across multiple documents, can significantly speed up the information-seeking process. Collecting human annotations for this application is prohibitively expensive due to long and complex exploration trajectories. We propose an agentic pipeline that automatically generates high quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. Our pipeline, SAGE, consists of a data generator which proposes QA pairs and a search agent which attempts to solve the generated question and provide execution feedback for the data generator. The two components interact over multiple rounds to iteratively refine the question-answer pairs until they satisfy the target difficulty level. Our intrinsic evaluation shows SAGE generates questions that require diverse reasoning strategies, while significantly increases the correctness and difficulty of the generated data. Our extrinsic evaluation demonstrates up to 23% relative performance gain on popular deep search benchmarks by training deep search agents with our synthetic data. Additional experiments show that agents trained on our data can adapt from fixed-corpus retrieval to Google Search at inference time, without further training.

Figures (10)

• Figure 1: Illustration of our proposed method: (1) we first employ a data generator ($A_{gen}$) to generate an initial question, answer (QA) pair with a target number of search steps $S$ for difficulty control. Our framework then (2) collect execution traces from search agent ($A_{search}$) and (3) provide the execution feedback for the data generator agent to re-generate a QA pair. In this example, the initial QA pair generated does not satisfy the target step S=4. After re-generation with search agent's feedback, the new QA pair is both correct and difficult. Noticeably, the question complexity increases from $\mathbf{q_0}$ to $\mathbf{q_1}$.
• Figure 2: Percentage of correct and difficult data for target steps of 3 to 7 for various methods. Updating the question with execution feedback further improves the data quality compared to resampling, particularly with more target search steps.
• Figure 3: Distribution of reasoning types required to answer questions in Musique and our generated dataset. Our data spans a broad set of reasoning types, resulting in a more balanced distribution compared to Musique.
• Figure 4: Prompt used for initial data generator.
• Figure 5: Prompt used for search agent.