An empirical study to understand how students use ChatGPT for writing essays and how it affects their ownership
Andrew Jelson, Sang Won Lee
TL;DR
Problem: As LLMs become common in student writing, educators struggle to understand usage patterns and the effect on writing ownership. Approach: the authors propose an empirical study that embeds a ChatGPT-like assistant within a writing platform and records fine-grained writing activity, prompts, and responses as students answer a predefined ethics prompt. Contributions: (i) a taxonomy of prompts and usage patterns, (ii) analysis linking ChatGPT content to writing edits and perceived ownership via an exit survey, and (iii) guidance for teaching practices and tooling design to support responsible AI use in coursework. Significance: results will inform policy, assessment design, and the development of features in AI-assisted writing tools to align with educational goals.
Abstract
As large language models (LLMs) become more powerful and ubiquitous, systems like ChatGPT are increasingly used by students to help them with writing tasks. To better understand how these tools are used, we investigate how students might use an LLM for essay writing, for example, to study the queries asked to ChatGPT and the responses that ChatGPT gives. To that end, we plan to conduct a user study that will record the user writing process and present them with the opportunity to use ChatGPT as an AI assistant. This study's findings will help us understand how these tools are used and how practitioners -- such as educators and essay readers -- should consider writing education and evaluation based on essay writing.