Synthesizing Agentic Data for Web Agents with Progressive Difficulty Enhancement Mechanisms

TL;DR

The paper tackles the challenge of training web agents capable of long-horizon reasoning by introducing ProgSearch, a two-pronged data synthesis pipeline that progressively escalates task difficulty using a frontier baseline web agent for validation. The top-down component builds a tree-of-facts to generate novel, multi-hop QA pairs, while the bottom-up component anchors on rare entities and hardens questions through iterative solver feedback. A consolidated filtering stage enforces strict quality and factuality criteria, yielding a compact yet high-quality dataset that supports distillation-based fine-tuning. Across multiple web benchmarks, models trained on ProgSearch achieve stronger performance and exhibit more diverse tool-use behavior than those trained on larger baselines, demonstrating that carefully crafted, difficulty-aligned synthetic data can outperform larger, less-targeted datasets. The work shows that controlled data synthesis coupled with robust filtering can meaningfully improve long-horizon web-agent capabilities with potentially lower data requirements.

Abstract

Web-based 'deep research' agents aim to solve complex question - answering tasks through long-horizon interactions with online tools. These tasks remain challenging, as the underlying language models are often not optimized for long-horizon reasoning and exploration. Prior work has proposed workflows for constructing instruction-tuning datasets, often leveraging knowledge graphs. However, such methods typically lack fine-grained control over difficulty and quality, yielding synthetic data that falls short of capturing the complexity required for long-horizon reasoning. Furthermore, many studies conflate data and training effects by comparing models trained under different optimization recipes, making it difficult to isolate and evaluate the effectiveness of the data itself. We introduce a two-pronged data synthesis pipeline that generates question - answer pairs by progressively increasing task complexity until a frontier baseline web agent fails. The baseline agent plays multiple roles in this process: attempting the questions, validating factuality, checking for alternative answers, and enforcing filtering. To evaluate the effectiveness of our synthesis methods, we adopt a controlled training setup based on distillation from strong web agents. Experiments across multiple web-based benchmarks show that our dataset - despite being smaller - enables the training of more effective web agents than existing datasets. In particular, our data exhibits twice the diversity in tool-use actions, allowing models trained on it to achieve stronger performance while avoiding repetitive tool-calling behaviors.

Figures (7)

• Figure 1: Overview of our ProgSearch two-pronged synthetic data generation pipeline. In the top-down approach, a tree-of-facts is constructed from a seed entity and complex QA pairs are synthesized via an iterative refinement method. The bottom-up approach selects a rare entity and iteratively generates a multi-constraint question about that entity. Synthesized data is then passed through quality and uniqueness filtering process to rule out problematic samples.
• Figure 2: Top-down synthetic data generation with tree-of-facts.
• Figure 3: Bottom-up data synthesis process with rare entity anchor.
• Figure 4: Broad category distribution ProgSearch, ASearcher, and TaskCraft datasets.
• Figure : BuildTreeOfFacts: Recurisively build a tree of facts, given a seed $F_0=E_0$