Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeepInnovator: Triggering the Innovative Capabilities of LLMs

Tianyu Fan, Fengji Zhang, Yuxiang Zheng, Bei Chen, Xinyao Niu, Chengen Huang, Junyang Lin, Chao Huang

TL;DR

DeepInnovator presents a training framework to trigger genuine innovative capabilities in LLMs by grounding idea generation in automatically extracted, structured knowledge from vast literature and training via a Next Idea Prediction task. It combines a data-synthesis pipeline that compresses and organizes prior work with a process-oriented RL objective and a decoupled reward/Comment mechanism to encourage progressive ideation while curbing reward hacking. Empirical results show DeepInnovator-14B outperforms untrained baselines across multiple dimensions and generalizes to out-of-distribution domains, sometimes rivaling state-of-the-art models. By open-sourcing datasets and code, the work offers a scalable pathway for building autonomous research agents with forward-looking, innovative capabilities.

Abstract

The application of Large Language Models (LLMs) in accelerating scientific discovery has garnered increasing attention, with a key focus on constructing research agents endowed with innovative capability, i.e., the ability to autonomously generate novel and significant research ideas. Existing approaches predominantly rely on sophisticated prompt engineering and lack a systematic training paradigm. To address this, we propose DeepInnovator, a training framework designed to trigger the innovative capability of LLMs. Our approach comprises two core components. (1) ``Standing on the shoulders of giants''. We construct an automated data extraction pipeline to extract and organize structured research knowledge from a vast corpus of unlabeled scientific literature. (2) ``Conjectures and refutations''. We introduce a ``Next Idea Prediction'' training paradigm, which models the generation of research ideas as an iterative process of continuously predicting, evaluating, and refining plausible and novel next idea. Both automatic and expert evaluations demonstrate that our DeepInnovator-14B significantly outperforms untrained baselines, achieving win rates of 80.53\%-93.81\%, and attains performance comparable to that of current leading LLMs. This work provides a scalable training pathway toward building research agents with genuine, originative innovative capability, and will open-source the dataset to foster community advancement. Source code and data are available at: https://github.com/HKUDS/DeepInnovator.

DeepInnovator: Triggering the Innovative Capabilities of LLMs

TL;DR

DeepInnovator presents a training framework to trigger genuine innovative capabilities in LLMs by grounding idea generation in automatically extracted, structured knowledge from vast literature and training via a Next Idea Prediction task. It combines a data-synthesis pipeline that compresses and organizes prior work with a process-oriented RL objective and a decoupled reward/Comment mechanism to encourage progressive ideation while curbing reward hacking. Empirical results show DeepInnovator-14B outperforms untrained baselines across multiple dimensions and generalizes to out-of-distribution domains, sometimes rivaling state-of-the-art models. By open-sourcing datasets and code, the work offers a scalable pathway for building autonomous research agents with forward-looking, innovative capabilities.

Abstract

The application of Large Language Models (LLMs) in accelerating scientific discovery has garnered increasing attention, with a key focus on constructing research agents endowed with innovative capability, i.e., the ability to autonomously generate novel and significant research ideas. Existing approaches predominantly rely on sophisticated prompt engineering and lack a systematic training paradigm. To address this, we propose DeepInnovator, a training framework designed to trigger the innovative capability of LLMs. Our approach comprises two core components. (1) ``Standing on the shoulders of giants''. We construct an automated data extraction pipeline to extract and organize structured research knowledge from a vast corpus of unlabeled scientific literature. (2) ``Conjectures and refutations''. We introduce a ``Next Idea Prediction'' training paradigm, which models the generation of research ideas as an iterative process of continuously predicting, evaluating, and refining plausible and novel next idea. Both automatic and expert evaluations demonstrate that our DeepInnovator-14B significantly outperforms untrained baselines, achieving win rates of 80.53\%-93.81\%, and attains performance comparable to that of current leading LLMs. This work provides a scalable training pathway toward building research agents with genuine, originative innovative capability, and will open-source the dataset to foster community advancement. Source code and data are available at: https://github.com/HKUDS/DeepInnovator.
Paper Structure (23 sections, 2 equations, 3 figures, 8 tables)

This paper contains 23 sections, 2 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Formulation for Innovation task
  4. Training Methodology of DeepInnovator
  5. Automated Data Extraction and Synthesis
  6. Reward Signal Design
  7. Decomposing Reward and Comment in Open-ended Tasks
  8. Experiments
  9. Experimental Settings
  10. Rubrics Evaluation Result
  11. Winrate Evaluation Result
  12. Expert Evaluation
  13. Conclusion
  14. Experiment Details.
  15. Datasets.
  16. ...and 8 more sections

Figures (3)

  • Figure 1: The DeepInnovator Framework. Top: We construct training data from arXiv through a carefully designed automated data extraction and synthesis pipeline (Sec. \ref{['sec:data']}). Bottom: We perform RL training via a meticulously designed next-idea prediction task (\ref{['sec:train_obj']}), coupled with a decoupled reward and critique mechanism (Sec. \ref{['sec:reward']}).
  • Figure 2: Evaluation results of ideas generated by eight models across six rubrics. DeepInnovator outperforms Qwen-14B-Instruct on all dimensions and demonstrates competitive capability against leading LLMs.
  • Figure 3: Comparison of win rates between ideas generated by DeepInnovator at each step and the initial idea (step 1). Our training method effectively enhances DeepInnovator's ability to refine research ideas.