Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery

Aniketh Garikaparthi, Manasi Patwardhan, Lovekesh Vig, Arman Cohan

TL;DR

This paper addresses accelerating scientific hypothesis generation by integrating a human-in-the-loop with large language models. It introduces IRIS, an open-source Interactive Research Ideation System that uses a three-agent architecture (Ideation, Review, Retrieval) and an adaptive Monte Carlo Tree Search framework to iteratively generate and refine research briefs, guided by fine-grained feedback and literature retrieval. The approach includes a formal MCTS process with states, actions, and the UCT selection criterion, along with memory and budget controls to balance exploration and exploitation. A user study across disciplines demonstrates improved hypothesis quality (average absolute score increase of ~0.5) and higher AUC-like ELO ratings (≈12 points) at depth 3, with strong user appreciation for steerability and feedback. Overall, IRIS offers a practical, open-source path toward human-AI co-creation in scientific ideation, enabling more controllable and interpretable LLM-assisted discovery.

Abstract

The rapid advancement in capabilities of large language models (LLMs) raises a pivotal question: How can LLMs accelerate scientific discovery? This work tackles the crucial first stage of research, generating novel hypotheses. While recent work on automated hypothesis generation focuses on multi-agent frameworks and extending test-time compute, none of the approaches effectively incorporate transparency and steerability through a synergistic Human-in-the-loop (HITL) approach. To address this gap, we introduce IRIS: Interactive Research Ideation System, an open-source platform designed for researchers to leverage LLM-assisted scientific ideation. IRIS incorporates innovative features to enhance ideation, including adaptive test-time compute expansion via Monte Carlo Tree Search (MCTS), fine-grained feedback mechanism, and query-based literature synthesis. Designed to empower researchers with greater control and insight throughout the ideation process. We additionally conduct a user study with researchers across diverse disciplines, validating the effectiveness of our system in enhancing ideation. We open-source our code at https://github.com/Anikethh/IRIS-Interactive-Research-Ideation-System

IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery

TL;DR

This paper addresses accelerating scientific hypothesis generation by integrating a human-in-the-loop with large language models. It introduces IRIS, an open-source Interactive Research Ideation System that uses a three-agent architecture (Ideation, Review, Retrieval) and an adaptive Monte Carlo Tree Search framework to iteratively generate and refine research briefs, guided by fine-grained feedback and literature retrieval. The approach includes a formal MCTS process with states, actions, and the UCT selection criterion, along with memory and budget controls to balance exploration and exploitation. A user study across disciplines demonstrates improved hypothesis quality (average absolute score increase of ~0.5) and higher AUC-like ELO ratings (≈12 points) at depth 3, with strong user appreciation for steerability and feedback. Overall, IRIS offers a practical, open-source path toward human-AI co-creation in scientific ideation, enabling more controllable and interpretable LLM-assisted discovery.

Abstract

The rapid advancement in capabilities of large language models (LLMs) raises a pivotal question: How can LLMs accelerate scientific discovery? This work tackles the crucial first stage of research, generating novel hypotheses. While recent work on automated hypothesis generation focuses on multi-agent frameworks and extending test-time compute, none of the approaches effectively incorporate transparency and steerability through a synergistic Human-in-the-loop (HITL) approach. To address this gap, we introduce IRIS: Interactive Research Ideation System, an open-source platform designed for researchers to leverage LLM-assisted scientific ideation. IRIS incorporates innovative features to enhance ideation, including adaptive test-time compute expansion via Monte Carlo Tree Search (MCTS), fine-grained feedback mechanism, and query-based literature synthesis. Designed to empower researchers with greater control and insight throughout the ideation process. We additionally conduct a user study with researchers across diverse disciplines, validating the effectiveness of our system in enhancing ideation. We open-source our code at https://github.com/Anikethh/IRIS-Interactive-Research-Ideation-System

Paper Structure

This paper contains 11 sections, 1 equation, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. IRIS
  4. Agent Architecture
  5. Monte Carlo Tree Search Framework
  6. Evaluation
  7. Experiment Setup
  8. User Study
  9. Results and Analysis
  10. Conclusion
  11. Review Taxonomy

Figures (4)

  • Figure 1: Human-in-the-loop Idea Generation with Monte-Carlo-Tree-Search. $\mathcal{G}$: Research Goal, $\mathcal{B}$: Research Brief
  • Figure 2: IRIS Platform Interface with (L) Retrieval Panel, (C) Chat Overview Panel, (R) Research Brief Panel
  • Figure 3: Iterative improvement in hypothesis quality within IRIS over interaction depth (up to depth 3). Interaction enhances both absolute scores and ELO ratings.
  • Figure 5: Top: Comparison of hypothesis quality generated by baseline methods (ChatGPT, ChatGPT+Search, Claude 3.5 Haiku, Gemini-2.0-Flash) using LLM-as-a-judge absolute scores and ELO ratings. Bottom: User Survey Feedback Form Questions.