Patterns and Purposes: A Cross-Journal Analysis of AI Tool Usage in Academic Writing
Ziyang Xu
TL;DR
The paper investigates how AI tools are used in scholarly writing by analyzing 168 AI usage declarations from 8,859 articles across 27 journal categories. It employs an embedded mixed-methods approach—content analysis, quantitative tests, and text mining—to identify tool types, usage purposes, and the influence of author language background and team composition. Key findings include a dominant role for ChatGPT (77%), with readability and grammar checking as the primary aims, and significant differences by native vs non-native English speakers and by international vs non-international teams, evidenced by $p = 0.0483$ and $p = 0.0012$. The study informs journal policy, highlights the need for AI literacy, and discusses practical implications for differentiating between polishing and content generation across the research lifecycle, while noting limitations such as sample size and reliance on self-reported usage.
Abstract
This study investigates the use of AI tools in academic writing through analysis of AI usage declarations in journals. Using a mixed-methods approach combining content analysis, statistical analysis, and text mining, this research analyzed 168 AI declarations from 8,859 articles across 27 categories. Results show that ChatGPT dominates academic writing assistance (77% usage), with significant differences in tool usage between native and non-native English speakers (p = 0.0483) and between international and non-international teams (p = 0.0012). The study reveals that improving readability (51%) and grammar checking (22%) are the primary purposes of AI tool usage. These findings provide insights for journal policy development and understanding the evolving role of AI in academic writing.