From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems

TL;DR

This paper surveys AI-driven research-support systems organized around hypothesis formulation, validation, and manuscript publication. It presents a taxonomy and analyzes methods for knowledge synthesis, hypothesis generation, claim verification, theorem proving, experiment design, and automated writing and review. It also inventories benchmarks and tools, discusses ethical considerations, and outlines future directions, including integration of diverse tasks and reasoning-oriented LLMs. The work highlights that the field is still in early stages, with ongoing challenges in generalizability, evaluation standards, and system reliability, but holds potential to transform how researchers generate, validate, and publish scientific work.

Abstract

Research is a fundamental process driving the advancement of human civilization, yet it demands substantial time and effort from researchers. In recent years, the rapid development of artificial intelligence (AI) technologies has inspired researchers to explore how AI can accelerate and enhance research. To monitor relevant advancements, this paper presents a systematic review of the progress in this domain. Specifically, we organize the relevant studies into three main categories: hypothesis formulation, hypothesis validation, and manuscript publication. Hypothesis formulation involves knowledge synthesis and hypothesis generation. Hypothesis validation includes the verification of scientific claims, theorem proving, and experiment validation. Manuscript publication encompasses manuscript writing and the peer review process. Furthermore, we identify and discuss the current challenges faced in these areas, as well as potential future directions for research. Finally, we also offer a comprehensive overview of existing benchmarks and tools across various domains that support the integration of AI into the research process. We hope this paper serves as an introduction for beginners and fosters future research. Resources have been made publicly available at https://github.com/zkzhou126/AI-for-Research.

Figures (6)

• Figure 1: Overview of AI for research. The framework consists of three stages: hypothesis formulation, hypothesis validation, and manuscript publication. In the hypothesis formulation stage, knowledge integration leads to the proposal of an initial hypothesis after a topic is identified. The hypothesis validation stage involves verifying the hypothesis from three perspectives to ensure its correctness and validity. Finally, the manuscript publication stage focuses on drafting and publishing the validated hypothesis.
• Figure 2: Taxonomy of Hypothesis Formulation, Hypothesis Validation and Manuscript Publication (This is a simplified version, full version in Figure \ref{['fig:taxonomy_full']}).
• Figure 3: This figure illustrates the hypothesis formulation process, consisting of two stages: knowledge synthesis and hypothesis generation, which together produce an initial hypothesis related to a specific topic.
• Figure 4: This figure illustrates the various perspectives for hypothesis validation during the hypothesis validation stage. A hypothesis is typically divided into scientific claims and theorems, with SCI-claim verification (scientific claim verification) and theorem proving ensuring theoretical correctness, while experiment validation assesses practical feasibility.
• Figure 5: This figure shows the transformation of a validated hypothesis into a publication, leveraging outputs from the hypothesis formulation and validation stages.