AI Does Not Alter Perceptions of Text Messages
N'yoma Diamond
TL;DR
This study investigates whether the belief that AI-assisted text composition was used alters recipient perceptions of text messages. In a controlled survey (n=26) with 18 pre-composed messages across six topics, messages were randomly labeled as AI-assisted, not AI-assisted, or unlabeled, and participants rated tone, clarity, and ability to convey intent. Across all analyses, there was no statistically significant effect of labeling on any perceptual dimension, though small mean differences suggested potential trends that larger samples might detect. The findings challenge assumptions that AI-generated or AI-assisted text is inherently perceived more negatively, supporting the viability of AI-MC tools without counter-productive perceptual consequences. Limitations include a relatively homogeneous, STEM-focused sample and a single-message context, indicating a need for broader, on-going studies to validate generalizability and explore long-term effects.
Abstract
For many people, anxiety, depression, and other social and mental factors can make composing text messages an active challenge. To remedy this problem, large language models (LLMs) may yet prove to be the perfect tool to assist users that would otherwise find texting difficult or stressful. However, despite rapid uptake in LLM usage, considerations for their assistive usage in text message composition have not been explored. A primary concern regarding LLM usage is that poor public sentiment regarding AI introduces the possibility that its usage may harm perceptions of AI-assisted text messages, making usage counter-productive. To (in)validate this possibility, we explore how the belief that a text message did or did not receive AI assistance in composition alters its perceived tone, clarity, and ability to convey intent. In this study, we survey the perceptions of 26 participants on 18 randomly labeled pre-composed text messages. In analyzing the participants' ratings of message tone, clarity, and ability to convey intent, we find that there is no statistically significant evidence that the belief that AI is utilized alters recipient perceptions. This provides hopeful evidence that LLM-based text message composition assistance can be implemented without the risk of counter-productive outcomes.