Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking

TL;DR

This study investigates how Large Language Model (LLM) assistance affects human creativity across divergent and convergent thinking. Using two pre-registered randomized experiments with 1,100 participants, participants engaged in exposure rounds with three AI conditions (no aid, direct AI answers, or coach-like guidance) and then completed the same tasks unassisted to measure residual effects. Findings show that AI assistance yields short-term gains during exposure but can hinder unaided performance afterward, with divergent thinking more prone to skepticism and homogenization and convergent thinking showing nuanced responses to coaching versus direct answers. The work highlights design implications for sustained human creativity in AI-assisted workflows and calls for coach-like systems that enhance long-term cognitive diversity rather than promoting over-reliance. Overall, the results suggest careful calibration of AI support to preserve and promote independent creative abilities in the age of generative AI and LLMs.

Abstract

Large language models are transforming the creative process by offering unprecedented capabilities to algorithmically generate ideas. While these tools can enhance human creativity when people co-create with them, it's unclear how this will impact unassisted human creativity. We conducted two large pre-registered parallel experiments involving 1,100 participants attempting tasks targeting the two core components of creativity, divergent and convergent thinking. We compare the effects of two forms of large language model (LLM) assistance -- a standard LLM providing direct answers and a coach-like LLM offering guidance -- with a control group receiving no AI assistance, and focus particularly on how all groups perform in a final, unassisted stage. Our findings reveal that while LLM assistance can provide short-term boosts in creativity during assisted tasks, it may inadvertently hinder independent creative performance when users work without assistance, raising concerns about the long-term impact on human creativity and cognition.

Figures (11)

• Figure 1: Schematic of design for Experiment 1 on divergent thinking.
• Figure 2: Interface used in the divergent thinking experiment across all 3 exposure round conditions and test rounds.
• Figure 3: Plots of the Alternate Uses Task ideas for the various divergent thinking dimensions (Segmented by phase and/or LLM response type). The left figure shows idea originality scores, the middle figure indicates idea fluency, and the right figure presents participant creative flexibility.
• Figure 4: Plots of the average individual- (left) and group-level (right) median diversity, segmented by experiment conditions and phases. Higher values denote more difference between ideas. Error bars represent $\pm$ one standard error of mean.
• Figure 5: Plots of subjective measures collected before and after participants completed the Alternate Uses Task. The left figure shows participants' change in self perceived creativity levels (Based on how many % of humans they felt they were more creative than), the middle figure indicates how their feelings towards the increased use of AI computer programs in daily life changed (Between More concerned than excited/More excited than concerned/Equally excited and concerned), and the right figure presents how much difficulty they had in coming up with ideas for the test object, all segmented by the three LLM response types.