LDC: Learning to Generate Research Idea with Dynamic Control

TL;DR

This paper tackles automated generation of research ideas by framing high quality ideas along three subdimensions: novelty, feasibility, and effectiveness. It introduces a two-stage pipeline—supervised fine-tuning on paper–idea pairs followed by controllable reinforcement learning guided by multi-dimensional reward models—along with dimensional controllers and a dynamic sentence-level decoding mechanism. Evaluation on a large, real-world corpus from ICLR/NeurIPS, complemented by retrieval-augmented automatic scoring and expert human judgments, demonstrates that dynamic control across multiple dimensions yields more balanced and high-quality ideas than baselines. The approach provides a practical path toward more controllable and reliable AI-driven ideation in scientific research, with implications for speeding up the research cycle while maintaining expert-aligned quality.

Abstract

Recent advancements in large language models (LLMs) have demonstrated their potential in automating the scientific research ideation. Existing approaches primarily focus on prompting techniques, often producing ideas misaligned with expert standards - novelty, feasibility, and effectiveness, which are widely recognized by the research community as the three key subdimensions of high-quality ideas. Also, balancing these dimensions remains challenging due to their inherent trade-offs. To address these limitations, we propose the first framework that employs a two-stage approach combining Supervised Fine-Tuning (SFT) and controllable Reinforcement Learning (RL) for the task. In the SFT stage, the model learns foundational patterns from pairs of research papers and their corresponding follow-up ideas. In the RL stage, multi-dimensional reward models guided by fine-grained feedback evaluate and optimize the model across key dimensions. During inference, dimensional controllers coordinated by a sentence-level decoder enable dynamic context-aware steering of the idea generation process. Our framework provides a balanced approach to research idea generation, achieving high-quality outcomes in the experiment by dynamically navigating the trade-offs among novelty, feasibility, and effectiveness.

Figures (8)

• Figure 1: Research idea generation from research papers. Each idea is measured across the dimensions of novelty, feasibility, and effectiveness.
• Figure 2: The learning framework with dynamic control across 3 dimensions. Generated research ideas are assessed by corresponding reward models, which provide scores for each dimension. These scores guide the fine-tuning process during reinforcement learning, optimizing both the idea proposer and the corresponding dimensional control parameters to enhance the quality of idea generation. Fires denote weight changes during the process.
• Figure 3: Decoding RNN dynamically steers the dimensions for a balanced and context-aware generation. The process starts with $\epsilon^{0}$ and predicts the control weights for the next sentence condition on the generated context.
• Figure 4: Dimensional variation w.r.t. normalized sentence position (1-10 according to idea length).
• Figure 5: Rating and topic statistics of our dataset.