Reactive Writers: How Co-Writing with AI Changes How We Engage with Ideas

TL;DR

An evaluation-first, suggestion-led writing practice that departs substantially from conventional composing in the presence of AI assistance and is highly vulnerable to AI-induced biases and opinion shifts is described.

Abstract

Emerging experimental evidence shows that writing with AI assistance can change both the views people express in writing and the opinions they hold afterwards. Yet, we lack substantive understanding of procedural and behavioral changes in co-writing with AI that underlie the observed opinion-shaping power of AI writing tools. We conducted a mixed-methods study, combining retrospective interviews with 19 participants about their AI co-writing experience with a quantitative analysis tracing engagement with ideas and opinions in 1{,}291 AI co-writing sessions. Our analysis shows that engaging with the AI's suggestions -- reading them and deciding whether to accept them -- becomes a central activity in the writing process, taking away from more traditional processes of ideation and language generation. As writers often do not complete their own ideation before engaging with suggestions, the suggested ideas and opinions seeded directions that writers then elaborated on. At the same time, writers did not notice the AI's influence and felt in full control of their writing, as they -- in principle -- could always edit the final text. We term this shift \textit{Reactive Writing}: an evaluation-first, suggestion-led writing practice that departs substantially from conventional composing in the presence of AI assistance and is highly vulnerable to AI-induced biases and opinion shifts.

Figures (8)

• Figure 1: Screenshot of the experimental writing platform showing AI suggestions.
• Figure 2: Simplified overview of the topic classification pipeline. Step 1 (topic discovery) shows how initial topics were identified from user-written and accepted text using gpt-3.5-turbo, with each topic receiving a description to enable clustering. Step 2 (topic assignment) shows how text was split into sub-sentences based on topic separation, with each sub-sentence assigned by gpt-3.5-turbo to one of the final topics derived from hierarchical clustering.
• Figure 3: A model of the Reactive Writing process. In this model, a new AI suggestion triggers Step 1: attention capture, where a writer's attention is redirected from ideation to a new suggestion. In Step 2: agreement-governed inclusion, the writer immediately begins to evaluate whether they wish to accept or reject the text in their writing, with a bias towards accepting suggestions. The diagram lists the main reasons to include or reject the text. When text has been accepted, writers turn to Step 3: post-hoc personalization, where they incorporate and adapt the suggestions to their own writing.
• Figure 4: Writing with AI did not substantially reduce the time participants spent on the essay.Mean task time with 95% confidence intervals in N = 1,291 writing sessions. When writing without AI assistance, participants, on average, wrote for 269 seconds. Participants writing with AI assistance spent about 248-250 seconds on their essay.
• Figure 5: Frequency of topics in the AI suggestions (left) and final text (right) when writing with a positive or critical AI assistant.Grey bars indicate control group frequencies for reference. N=1,291, 95% confidence intervals shown in black. The AI assistant's suggestions (left panel) showed a clear tendency towards certain topics, with many suggestions related to global connectivity in the positive AI treatment group, and many suggestions related to addiction, bullying, and loneliness in the critical AI treatment group. Participants' written text (right panel) inherited the topical biases of the suggestions to a substantial extent.