Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Divergent-Convergent Thinking in Large Language Models for Creative Problem Generation

Manh Hung Nguyen, Adish Singla

TL;DR

This work addresses the lack of diversity in LLM-generated educational problems caused by the Artificial Hivemind by introducing CreativeDC, a two-phase prompting framework that imposes divergent thinking followed by convergent thinking to expand the ideation space before constraint satisfaction. By incorporating persona simulation, CreativeDC further broadens perspective and output variety. Empirical evaluation across 20 contexts shows that CreativeDC substantially improves lexical and semantic diversity and novelty while maintaining high utility, with the Vendi score indicating a larger effective number of distinct problems as K grows. The approach demonstrates scalable gains and has potential applicability beyond programming problems to other creative domains, offering a practical method to counteract premature convergence in LLM reasoning.

Abstract

Large language models (LLMs) have significant potential for generating educational questions and problems, enabling educators to create large-scale learning materials. However, LLMs are fundamentally limited by the ``Artificial Hivemind'' effect, where they generate similar responses within the same model and produce homogeneous outputs across different models. As a consequence, students may be exposed to overly similar and repetitive LLM-generated problems, which harms diversity of thought. Drawing inspiration from Wallas's theory of creativity and Guilford's framework of divergent-convergent thinking, we propose CreativeDC, a two-phase prompting method that explicitly scaffolds the LLM's reasoning into distinct phases. By decoupling creative exploration from constraint satisfaction, our method enables LLMs to explore a broader space of ideas before committing to a final problem. We evaluate CreativeDC for creative problem generation using a comprehensive set of metrics that capture diversity, novelty, and utility. The results show that CreativeDC achieves significantly higher diversity and novelty compared to baselines while maintaining high utility. Moreover, scaling analysis shows that CreativeDC generates a larger effective number of distinct problems as more are sampled, increasing at a faster rate than baseline methods.

Divergent-Convergent Thinking in Large Language Models for Creative Problem Generation

TL;DR

This work addresses the lack of diversity in LLM-generated educational problems caused by the Artificial Hivemind by introducing CreativeDC, a two-phase prompting framework that imposes divergent thinking followed by convergent thinking to expand the ideation space before constraint satisfaction. By incorporating persona simulation, CreativeDC further broadens perspective and output variety. Empirical evaluation across 20 contexts shows that CreativeDC substantially improves lexical and semantic diversity and novelty while maintaining high utility, with the Vendi score indicating a larger effective number of distinct problems as K grows. The approach demonstrates scalable gains and has potential applicability beyond programming problems to other creative domains, offering a practical method to counteract premature convergence in LLM reasoning.

Abstract

Large language models (LLMs) have significant potential for generating educational questions and problems, enabling educators to create large-scale learning materials. However, LLMs are fundamentally limited by the ``Artificial Hivemind'' effect, where they generate similar responses within the same model and produce homogeneous outputs across different models. As a consequence, students may be exposed to overly similar and repetitive LLM-generated problems, which harms diversity of thought. Drawing inspiration from Wallas's theory of creativity and Guilford's framework of divergent-convergent thinking, we propose CreativeDC, a two-phase prompting method that explicitly scaffolds the LLM's reasoning into distinct phases. By decoupling creative exploration from constraint satisfaction, our method enables LLMs to explore a broader space of ideas before committing to a final problem. We evaluate CreativeDC for creative problem generation using a comprehensive set of metrics that capture diversity, novelty, and utility. The results show that CreativeDC achieves significantly higher diversity and novelty compared to baselines while maintaining high utility. Moreover, scaling analysis shows that CreativeDC generates a larger effective number of distinct problems as more are sampled, increasing at a faster rate than baseline methods.
Paper Structure (19 sections, 4 figures, 4 tables)

This paper contains 19 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Problem Setup
  3. Creative Problem Generation
  4. Metrics for Evaluating Creativity
  5. Diversity.
  6. Novelty.
  7. Utility.
  8. Objective
  9. Our Proposed Method: CreativeDC
  10. Motivation
  11. Two-phase Thinking Process
  12. Persona Simulation Augmentation
  13. Evaluation
  14. Dataset
  15. Methods and Models Evaluated
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Creative Problem Generation with CreativeDC. The input is a context containing a theme of interest and a programming concept for practicing. Our method instruct an LLM to apply a two-stage thinking process before generating a creative problem. (1) Divergent thinking phase: exploring diverse and novel ideas related to the theme. (2) Convergent thinking phase: narrowing down to one idea and align it with the constraints (required programming concept) in the given context. The output creative problem consists of a description, a test suite and a sample solution.
  • Figure 2: Semantic Diversity and Semantic Novelty distributions. Kernel density estimates show per-problem scores for each method. In the without persona setting, CreativeDC achieves 16.7% higher semantic diversity and 63.5% higher semantic novelty than CoT (all p < 0.001, Mann-Whitney U test). With persona simulation, CreativeDC maintains substantial advantages (8.5% diversity, 32.9% novelty over CoT). CreativeDC also exhibits lower variance, indicating more consistent performance.
  • Figure 3: Effective number of distinct problems when varying the number of problems $K$. For each method, we show the Vendi score DBLP:journals/tmlr/FriedmanD23 computed on all problems (dashed lines) and on problems with utility score of 1 only (solid lines). The shaded bands represent the standard error of the mean Vendi score across different contexts. In the without persona setting, CreativeDC outperforms CoT by 24.0% at $K=10$ and 72.0% at $K=100$, achieving faster growth in the effective number of distinct problems compared to CoT. With persona simulation, CreativeDC maintains its diversity advantage, demonstrating effective diversity at scale.
  • Figure 4: Performance of CreativeDC across contexts. Everyday theme such as "Cooking" achieves highest utility but lowest diversity and novelty, while more creative themes enable higher diversity and novelty. Simpler programming concepts (e.g., Variables) allows generating more novel problems.