Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent

Xinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang, Qiuchen Wang, Ruixue Ding, Chenxi Wang, Jialong Wu, Yida Zhao, Kuan Li, Yong Jiang, Pengjun Xie, Fei Huang, Jingren Zhou

TL;DR

WebWatcher presents a vision-language deep research agent that integrates multi-tool reasoning with external web and computational tools to tackle complex multimodal information-seeking tasks. It introduces BrowseComp-VL, a large-scale, automated VQA benchmark grounded in authentic visual content and knowledge-rich multi-hop queries, and a fully automated trajectory generation pipeline for high-quality supervision. The approach combines SFT cold-start and GRPO reinforcement learning to achieve strong performance across multiple challenging VL benchmarks, demonstrating the value of tool-augmented planning and visual grounding. Collectively, the work advances autonomous multimodal reasoning and sets a scalable framework for future multimodal deep research agents.

Abstract

Web agents such as Deep Research have demonstrated superhuman cognitive abilities, capable of solving highly challenging information-seeking problems. However, most research remains primarily text-centric, overlooking visual information in the real world. This makes multimodal Deep Research highly challenging, as such agents require much stronger reasoning abilities in perception, logic, knowledge, and the use of more sophisticated tools compared to text-based agents. To address this limitation, we introduce WebWatcher, a multi-modal Agent for Deep Research equipped with enhanced visual-language reasoning capabilities. It leverages high-quality synthetic multimodal trajectories for efficient cold start training, utilizes various tools for deep reasoning, and further enhances generalization through reinforcement learning. To better evaluate the capabilities of multimodal agents, we propose BrowseComp-VL, a benchmark with BrowseComp-style that requires complex information retrieval involving both visual and textual information. Experimental results show that WebWatcher significantly outperforms proprietary baseline, RAG workflow and open-source agents in four challenging VQA benchmarks, which paves the way for solving complex multimodal information-seeking tasks.

WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent

TL;DR

WebWatcher presents a vision-language deep research agent that integrates multi-tool reasoning with external web and computational tools to tackle complex multimodal information-seeking tasks. It introduces BrowseComp-VL, a large-scale, automated VQA benchmark grounded in authentic visual content and knowledge-rich multi-hop queries, and a fully automated trajectory generation pipeline for high-quality supervision. The approach combines SFT cold-start and GRPO reinforcement learning to achieve strong performance across multiple challenging VL benchmarks, demonstrating the value of tool-augmented planning and visual grounding. Collectively, the work advances autonomous multimodal reasoning and sets a scalable framework for future multimodal deep research agents.

Abstract

Web agents such as Deep Research have demonstrated superhuman cognitive abilities, capable of solving highly challenging information-seeking problems. However, most research remains primarily text-centric, overlooking visual information in the real world. This makes multimodal Deep Research highly challenging, as such agents require much stronger reasoning abilities in perception, logic, knowledge, and the use of more sophisticated tools compared to text-based agents. To address this limitation, we introduce WebWatcher, a multi-modal Agent for Deep Research equipped with enhanced visual-language reasoning capabilities. It leverages high-quality synthetic multimodal trajectories for efficient cold start training, utilizes various tools for deep reasoning, and further enhances generalization through reinforcement learning. To better evaluate the capabilities of multimodal agents, we propose BrowseComp-VL, a benchmark with BrowseComp-style that requires complex information retrieval involving both visual and textual information. Experimental results show that WebWatcher significantly outperforms proprietary baseline, RAG workflow and open-source agents in four challenging VQA benchmarks, which paves the way for solving complex multimodal information-seeking tasks.

Paper Structure

This paper contains 41 sections, 5 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Data Preparation
  3. Data Overview
  4. VQA Pairs Construction
  5. Generate QA
  6. Level 1
  7. Level 2
  8. QA-to-VQA Conversion
  9. Visual Context Construction for VQA
  10. Entity Masking and Question Transformation
  11. Quality Control
  12. Trajectory Generation and Post-Training
  13. Automated Generation of Reasoning Trajectories
  14. Multimodal Tools
  15. Automated Trajectory Annotation
  16. ...and 26 more sections

Figures (7)

  • Figure 1: Overall performance of WebWatcher compares to other models across four benchmarks. All other models are equipped with RAG workflow.
  • Figure 2: Comparison of VL reasoning agents. WebWatcher resolves the GAIA case that defeats either vision-only reasoning or search-based agents, demonstrating the strength of multi-tool integration and in-depth reasoning generalization.
  • Figure 3: Domain Distribution for Level 1 and Level 2.
  • Figure 4: Data generation pipelines.
  • Figure 5: The percentage of tool calls in the four benchmarks. The height of each bar denotes the fraction of total calls made to that tool within the corresponding benchmark.
  • ...and 2 more figures