Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL

Yi Yao, He Zhu, Piaohong Wang, Jincheng Ren, Xinlong Yang, Qianben Chen, Xiaowan Li, Dingfeng Shi, Jiaxian Li, Qiexiang Wang, Sinuo Wang, Xinpeng Liu, Jiaqi Wu, Minghao Liu, Wangchunshu Zhou

TL;DR

This work tackles the data gap limiting open-source LLMs by introducing O-Researcher, a multi-agent data-synthesis framework that end-to-end simulates tool-enabled research reasoning to produce high-quality instructional data. It then trains open-source models in two stages—supervised fine-tuning on synthetic trajectories, followed by reinforcement learning from AI feedback with a carefully designed reward structure—to maximize alignment and capability. Across major deep-research benchmarks, O-Researcher variants achieve new open-weight state-of-the-art, with RL further improving factual grounding and citation quality. The approach demonstrates a scalable path for advancing open-source LLMs without proprietary data, enabling robust, verifiable research reporting and strong performance on complex, long-horizon tasks.

Abstract

The performance gap between closed-source and open-source large language models (LLMs) is largely attributed to disparities in access to high-quality training data. To bridge this gap, we introduce a novel framework for the automated synthesis of sophisticated, research-grade instructional data. Our approach centers on a multi-agent workflow where collaborative AI agents simulate complex tool-integrated reasoning to generate diverse and high-fidelity data end-to-end. Leveraging this synthesized data, we develop a two-stage training strategy that integrates supervised fine-tuning with a novel reinforcement learning method, designed to maximize model alignment and capability. Extensive experiments demonstrate that our framework empowers open-source models across multiple scales, enabling them to achieve new state-of-the-art performance on the major deep research benchmark. This work provides a scalable and effective pathway for advancing open-source LLMs without relying on proprietary data or models.

O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL

TL;DR

This work tackles the data gap limiting open-source LLMs by introducing O-Researcher, a multi-agent data-synthesis framework that end-to-end simulates tool-enabled research reasoning to produce high-quality instructional data. It then trains open-source models in two stages—supervised fine-tuning on synthetic trajectories, followed by reinforcement learning from AI feedback with a carefully designed reward structure—to maximize alignment and capability. Across major deep-research benchmarks, O-Researcher variants achieve new open-weight state-of-the-art, with RL further improving factual grounding and citation quality. The approach demonstrates a scalable path for advancing open-source LLMs without proprietary data, enabling robust, verifiable research reporting and strong performance on complex, long-horizon tasks.

Abstract

The performance gap between closed-source and open-source large language models (LLMs) is largely attributed to disparities in access to high-quality training data. To bridge this gap, we introduce a novel framework for the automated synthesis of sophisticated, research-grade instructional data. Our approach centers on a multi-agent workflow where collaborative AI agents simulate complex tool-integrated reasoning to generate diverse and high-fidelity data end-to-end. Leveraging this synthesized data, we develop a two-stage training strategy that integrates supervised fine-tuning with a novel reinforcement learning method, designed to maximize model alignment and capability. Extensive experiments demonstrate that our framework empowers open-source models across multiple scales, enabling them to achieve new state-of-the-art performance on the major deep research benchmark. This work provides a scalable and effective pathway for advancing open-source LLMs without relying on proprietary data or models.
Paper Structure (33 sections, 3 equations, 4 figures, 4 tables)

This paper contains 33 sections, 3 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. High-Quality Trajectory Generation for SFT
  5. Producing Trajectory with Parallel Execution.
  6. Query Synthesis and Trajectory Collection.
  7. Quality Assurance Pipeline
  8. Structured Data Representation
  9. Reinforcement Learning from AI Feedback (RLAIF)
  10. Preference Data Curation.
  11. Reward Function Design.
  12. Base Quality Reward
  13. Tool-Usage Reward
  14. Formatting Reward
  15. Final Reward.
  16. ...and 18 more sections

Figures (4)

  • Figure 1: The report generation process of O-Researcher. A query is broken down into multiple sub-queries, which are then independently and parallelized by different agents through tool-integrated reasoning to generate sub-query reports. These sub-query reports are then aggregated through the summarizer agent to generate the final report. All traces and reports of different sub-queries are concatenated as the supervised-training data for this query.
  • Figure 2: The deep research model training process of our work, which has an SFT stage and an RL stage.
  • Figure 3: Qualitative case study of Parallel Execution. The complex adaptation task is decomposed into three distinct subtasks (Narrative, Logistics, Authenticity). Each subtask independently undergoes a Think-Search-Observation loop, retrieving high-granularity evidence (e.g., specific budget figures, casting numbers) before being synthesized into a comprehensive report. This contrasts with sequential methods that often fail to maintain such depth across multiple dimensions simultaneously.
  • Figure :