Who Owns Creativity and Who Does the Work? Trade-offs in LLM-Supported Research Ideation

TL;DR

This study investigates how different levels of human control over LLM agents affect creativity support, effort allocation, and perceived ownership in research ideation. It introduces an end-to-end agentic system with three roles—Ideator, Writer, and Evaluator—and three supervision levels—Low, Medium, and Intensive—to elicit end-to-end proposal drafting behavior. Across a mixed-methods study with $N=54$ researchers, findings reveal non-linear creativity-support gains, a shift of labor toward idea verification, and ownership that is negotiated between humans and AI rather than predetermined by control level. The work advocates designing LLM-assisted ideation tools that empower researchers and preserve agency over strong ideas, while addressing attribution, responsibility, and the evolving role of researchers in AI-augmented discovery.

Abstract

LLM-based agents offer new potential to accelerate science and reshape research work. However, the quality of researcher contributions can vary significantly depending on human ability to steer agent behaviors. How can we best use these tools to augment scientific creativity without undermining aspects of contribution and ownership that drive research? To investigate this, we developed an agentic research ideation system integrating three roles -- Ideator, Writer, and Evaluator -- across three control levels -- Low, Medium, and Intensive. Our mixed-methods study with 54 researchers suggests three key findings in how LLM-based agents reshape scientific creativity: 1) perceived creativity support does not simply increase linearly with greater control; 2) human effort shifts from ideating to verifying ideas; and 3) ownership becomes a negotiated outcome between human and AI. Our findings suggest that LLM agent design should emphasize researcher empowerment, fostering a sense of ownership over strong ideas rather than reducing researchers to operating an automated AI-driven process.

Figures (8)

• Figure 1: Screenshots of the ResearchGuru system dashboard. Users primarily work in the Writing Editor, where they can both initiate functions and edit their proposals. The Search Sidebar and Evaluation Sidebar can be opened on the left and right sides of the dashboard, respectively.
• Figure 2: Screenshots of different agentic functions for the ResearchGuru system dashboard. Users can select and edit agent suggested seed ideas; write prompt for synthesis and revisions, or highlight texts for LLM edits in each proposal section; customize criteria, select and customize agent suggested improvements; and revert versions for each section and review agent steps of operations.
• Figure 3: The RAG system integrates a rewrite-retrieve-read framework Ma2023-yk with zero-shot dense retrieval Gao2023-eb.
• Figure 4: The agentic writing workflow involves multiple LLM calls, each generating distinct outputs through an internal chain-of-thought process. In our development, we employ two different LLMs: the Ideator and Writer roles share the same LLM, while the Evaluator utilizes a different LLM. The system allows human researchers to stop or intervene each agent with different controls.
• Figure 5: Changes in creativity support (weighted and unweighted scores) and effort measures across three control levels. The top panels present CSI scores. The bottom shows NASA-TLX scores. The left side of each panel displays aggregated scores, and the right break down scores by individual dimension. Spearman correlation coefficients ($\rho$) and corresponding p-values are indicated along the line.