SciPIP: An LLM-based Scientific Paper Idea Proposer

TL;DR

SciPIP introduces a framework to automatically propose scientific ideas by combining a semantically enriched literature database with a multi-granularity retrieval strategy and a dual-path idea generation process. It addresses limitations of keyword-based retrieval and abstract-level embeddings by leveraging background-level semantics, citation relationships, and structured paper summaries. Extensive NLP and CV experiments show SciPIP outperforms baselines in novelty, clarity, feasibility, and relevance, demonstrating its potential to aid researchers in generating high-quality ideas. The work provides a scalable pipeline for literature-informed ideation that could accelerate interdisciplinary innovation and knowledge synthesis.

Abstract

The rapid advancement of large language models (LLMs) has opened new possibilities for automating the proposal of innovative scientific ideas. This process involves two key phases: literature retrieval and idea generation. However, existing approaches often fall short due to their reliance on keyword-based search tools during the retrieval phase, which neglects crucial semantic information and frequently results in incomplete retrieval outcomes. Similarly, in the idea generation phase, current methodologies tend to depend solely on the internal knowledge of LLMs or metadata from retrieved papers, thereby overlooking significant valuable insights contained within the full texts. To address these limitations, we introduce SciPIP, an innovative framework designed to enhance the LLM-based proposal of scientific ideas through improvements in both literature retrieval and idea generation. Our approach begins with the construction of a comprehensive literature database that supports advanced retrieval based not only on keywords but also on semantics and citation relationships. This is complemented by the introduction of a multi-granularity retrieval algorithm aimed at ensuring more thorough and exhaustive retrieval results. For the idea generation phase, we propose a dual-path framework that effectively integrates both the content of retrieved papers and the extensive internal knowledge of LLMs. This integration significantly boosts the novelty, feasibility, and practical value of proposed ideas. Our experiments, conducted across various domains such as natural language processing and computer vision, demonstrate SciPIP's capability to generate a multitude of innovative and useful ideas. These findings underscore SciPIP's potential as a valuable tool for researchers seeking to advance their fields with groundbreaking concepts.

Figures (6)

• Figure 1: The pipeline of constructing the literature database. Paper sections are extracted via a PDF parser, summarized by an LLM, encoded, and stored in the database. A paper-keyword graph linking each paper to its keywords is also stored.
• Figure 2: The pipeline of SKC-based literature retrieval and literature clustering. Red words in the user's query are entity examples.
• Figure 3: The pipeline of SciPIP for idea proposal.
• Figure 4: Comparison between SciPIP and AI Scientist. Both frameworks takes 3 backgrounds as input and generate about 30 ideas. GPT-4o-mini is used during generation.
• Figure 5: The distribution of human ratings of SciPIP proposed NLP ideas. GPT-4o is used during generation.