Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs can Realize Combinatorial Creativity: Generating Creative Ideas via LLMs for Scientific Research

Tianyang Gu, Jingjin Wang, Zhihao Zhang, HaoHong Li

TL;DR

The paper addresses how to harness Large Language Models for scientific idea generation by grounding ideation in Boden's combinatorial creativity and the four P's. It introduces a two-phase agent framework that combines a cross-domain generalization-level retrieval system with a structured two-stage combinatorial process to generate ideas. Experiments on the OAG-Bench dataset show consistent improvements in alignment with real research developments, suggesting that theory-guided prompting and retrieval can enhance AI-assisted research. The work contributes a principled foundation for more transferable and impactful machine creativity, with clear directions for extending beyond combinatorial creativity toward exploratory and transformational creativity and broader evaluation frameworks.

Abstract

Scientific idea generation has been extensively studied in creativity theory and computational creativity research, providing valuable frameworks for understanding and implementing creative processes. However, recent work using Large Language Models (LLMs) for research idea generation often overlooks these theoretical foundations. We present a framework that explicitly implements combinatorial creativity theory using LLMs, featuring a generalization-level retrieval system for cross-domain knowledge discovery and a structured combinatorial process for idea generation. The retrieval system maps concepts across different abstraction levels to enable meaningful connections between disparate domains, while the combinatorial process systematically analyzes and recombines components to generate novel solutions. Experiments on the OAG-Bench dataset demonstrate our framework's effectiveness, consistently outperforming baseline approaches in generating ideas that align with real research developments (improving similarity scores by 7\%-10\% across multiple metrics). Our results provide strong evidence that LLMs can effectively realize combinatorial creativity when guided by appropriate theoretical frameworks, contributing both to practical advancement of AI-assisted research and theoretical understanding of machine creativity.

LLMs can Realize Combinatorial Creativity: Generating Creative Ideas via LLMs for Scientific Research

TL;DR

The paper addresses how to harness Large Language Models for scientific idea generation by grounding ideation in Boden's combinatorial creativity and the four P's. It introduces a two-phase agent framework that combines a cross-domain generalization-level retrieval system with a structured two-stage combinatorial process to generate ideas. Experiments on the OAG-Bench dataset show consistent improvements in alignment with real research developments, suggesting that theory-guided prompting and retrieval can enhance AI-assisted research. The work contributes a principled foundation for more transferable and impactful machine creativity, with clear directions for extending beyond combinatorial creativity toward exploratory and transformational creativity and broader evaluation frameworks.

Abstract

Scientific idea generation has been extensively studied in creativity theory and computational creativity research, providing valuable frameworks for understanding and implementing creative processes. However, recent work using Large Language Models (LLMs) for research idea generation often overlooks these theoretical foundations. We present a framework that explicitly implements combinatorial creativity theory using LLMs, featuring a generalization-level retrieval system for cross-domain knowledge discovery and a structured combinatorial process for idea generation. The retrieval system maps concepts across different abstraction levels to enable meaningful connections between disparate domains, while the combinatorial process systematically analyzes and recombines components to generate novel solutions. Experiments on the OAG-Bench dataset demonstrate our framework's effectiveness, consistently outperforming baseline approaches in generating ideas that align with real research developments (improving similarity scores by 7\%-10\% across multiple metrics). Our results provide strong evidence that LLMs can effectively realize combinatorial creativity when guided by appropriate theoretical frameworks, contributing both to practical advancement of AI-assisted research and theoretical understanding of machine creativity.

Paper Structure

This paper contains 19 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. Computational creativity
  4. Person
  5. Process
  6. Product
  7. Press
  8. Idea generation via large language models
  9. Combinatorial creativity agent framework
  10. Idea Extraction and Retrieval
  11. Combinatorial Process
  12. Experiments
  13. Quantitative Evaluation
  14. Experimental Setup
  15. Baseline and Metrics
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Combinatorial creativity agent core
  • Figure 2: Semi-strctured idea data format and the level-wise retrieval system
  • Figure 3: Comparative analysis of similarity scores between our framework and baseline. From left to right: (1) Scatter plot showing overall performance comparison with the equality line, (2) Bar chart comparing average similarities across three key metrics, (3-5) Line plots showing paper-by-paper comparison for design rationale, universal principle, and key mechanism similarities respectively. Our method consistently outperforms the baseline across all metrics, with particularly strong advantages in stability and high-end performance.