Nova: An Iterative Planning and Search Approach to Enhance Novelty and Diversity of LLM Generated Ideas

TL;DR

Nova addresses the problem of repetitive, limited-knowledge ideas from LLMs by introducing an iterative planning and search framework that actively retrieves external knowledge to enrich ideation. The method comprises seed idea generation, iterative planning/search, and detailed output generation, yielding substantially higher novelty and diversity than baselines. Automatic Swiss Tournament and human evaluations on 170 seed papers show Nova producing more unique, high-quality ideas and aligning with human judgments. These results suggest planning-driven external information retrieval can significantly enhance LLM-based scientific ideation and has practical implications for accelerating innovative research.

Abstract

Scientific innovation is pivotal for humanity, and harnessing large language models (LLMs) to generate research ideas could transform discovery. However, existing LLMs often produce simplistic and repetitive suggestions due to their limited ability in acquiring external knowledge for innovation. To address this problem, we introduce an enhanced planning and search methodology designed to boost the creative potential of LLM-based systems. Our approach involves an iterative process to purposely plan the retrieval of external knowledge, progressively enriching the idea generation with broader and deeper insights. Validation through automated and human assessments indicates that our framework substantially elevates the quality of generated ideas, particularly in novelty and diversity. The number of unique novel ideas produced by our framework is 3.4 times higher than without it. Moreover, our method outperforms the current state-of-the-art, generating at least 2.5 times more top-rated ideas based on 170 seed papers in a Swiss Tournament evaluation.

Figures (8)

• Figure 1: Nova's Performance.The Left: Comparison with the state-of-the-arts. Nova significantly outperforms other agents si2024llmsgeneratenovelresearchbaek2024researchagentiterativeresearchidealu2024aiscientistfullyautomated in generating high-quality ideas (Swiss Tournament Score is 5). The Right: The number of unique novel ideas at each iteration step. The iterative planning framework significantly enhances the generation of unique novel ideas, increasing by 3.4 times from the baseline.
• Figure 2: Nova Pipeline. The Pipeline includes initial seed idea generation, seed idea iteration, and idea completion. Upon receiving an input paper (i.e., seed paper), the LLM is prompted to generate initial seed ideas by utilizing related papers (including recent publications) and scientific discovery methods. After that, the generated ideas are revised according to the new knowledge acquired according to iterative planning and search. Finally, each idea is expanded with more detailed methods.
• Figure 3: Example of Planning-Driven Iterative Seed Idea Generation Process. This example highlights the planning-driven iterative seed idea iteration process. Starting from an initial concept, a detailed plan is formulated to guide the search for relevant literature and acquire up-to-date knowledge.
• Figure 4: Score distribution of different methods in Swiss Tournament. The results indicate that Nova not only generates more unique ideas but also produces a greater proportion of high-quality ideas. 619 and 2521 ideas generated by Nova are scored at 4 and 5, significantly surpassing the baseline methods.
• Figure 5: Non-Duplicate Percentage Comparison.