How Problematic Writer-AI Interactions (Rather than Problematic AI) Hinder Writers' Idea Generation

TL;DR

This paper addresses whether generative AI aids or hinders writers’ idea generation during constructive learning, arguing that outcomes depend on the interaction between the writer and the AI rather than AI type alone. It employs a four-step method: construct a reflective-writing task, deploy two AI probes (Socratic and auto-complete), collect think-aloud and log data as participants write, and compare AI-enhanced versus AI-hampered ideation. The key findings show that writer engagement and three interaction patterns—mindless echoing, premature/prolonged copyediting, and writer-initiated topic shifts—shape ideation outcomes, with human–AI co-ideation sessions producing the most ideas. The work has practical implications for designing AI writing assistants and informs educators and policymakers to foster cognitively active writer–AI interactions to maximize constructive learning.

Abstract

Writing about a subject enriches writers' understanding of that subject. This cognitive benefit of writing -- known as constructive learning -- is essential to how students learn in various disciplines. However, does this benefit persist when students write with generative AI writing assistants? Prior research suggests the answer varies based on the type of AI, e.g., auto-complete systems tend to hinder ideation, while assistants that pose Socratic questions facilitate it. This paper adds an additional perspective. Through a case study, we demonstrate that the impact of genAI on students' idea development depends not only on the AI but also on the students and, crucially, their interactions in between. Students who proactively explored ideas gained new ideas from writing, regardless of whether they used auto-complete or Socratic AI assistants. Those who engaged in prolonged, mindless copyediting developed few ideas even with a Socratic AI. These findings suggest opportunities in designing AI writing assistants, not merely by creating more thought-provoking AI, but also by fostering more thought-provoking writer-AI interactions.

Figures (4)

• Figure 1: A visualization of one writer's thought process when writing an academic article nguyen2024human. Contrary to what theoretical models might suggest flower_Hayes1981cognitive, in practice, writers switch among various cognitive and metacognitive tasks "in a seemingly haphazard fashion” during writing quinlan2012coordinatingwang2025scholawritedataset, making it hard to analyze AI's impact on their thought processes.
• Figure 2: Screenshot of the text editor Yang_coauthor_CHI22 and the Socratic AI writing assistant in use. Accepted AI suggestions/questions were displayed as an outline in the writing panel (left) and were invisible in the review panel (right).
• Figure 3: The paces of semantic expansion in students' writing (a proxy for their idea generation) over the course of various writing sessions. While writing with the Socratic AI assistant resulted in greater semantic expansion compared to using auto-complete (left), the difference is much less substantial than the effect of the writers' active thinking and ideation (right).
• Figure 4: When students mindlessly echoed AI suggestions, their idea generation decreased. The figure illustrates this by showing the paces of semantic expansion in a student's writing—a proxy for their idea generation—when they wrote with a typical auto-complete system (left) versus a modified auto-complete system that detects mindless echoing in real time and stops it by halting auto-complete suggestions for 2 minutes (right). When using the modified system, the student generated similar amount of new ideas in much shorter time. The gray areas in both figures represent periods of "mindless echoing".