LLMs as Academic Reading Companions: Extending HCI Through Synthetic Personae

TL;DR

The paper addresses whether large language models can serve as effective academic reading companions in higher education, weighing potential benefits against risks of overreliance and ethical concerns. It presents an exploratory, between-subjects study using Anthropic Claude.ai as an interactive reading assistant to scaffold comprehension of complex qualitative literature in two graduate design courses. The study collects quantitative surveys and qualitative interviews from 60 participants, analyzes recorded LLM dialogues, and includes follow-up interviews, with a power analysis supporting the sample size. Early results indicate improvements in reading comprehension and engagement for the Claude.ai group, but warn about possible overtrust, hallucinations, and biases, underscoring the need for responsible, ethically guided deployment. Overall, the work provides practical guidance for designing synthetic personae and informs policy and institutional strategies to maximize benefits while safeguarding student well-being.

Abstract

This position paper argues that large language models (LLMs) constitute promising yet underutilized academic reading companions capable of enhancing learning. We detail an exploratory study examining Claude from Anthropic, an LLM-based interactive assistant that helps students comprehend complex qualitative literature content. The study compares quantitative survey data and qualitative interviews assessing outcomes between a control group and an experimental group leveraging Claude over a semester across two graduate courses. Initial findings demonstrate tangible improvements in reading comprehension and engagement among participants using the AI agent versus unsupported independent study. However, there is potential for overreliance and ethical considerations that warrant continued investigation. By documenting an early integration of an LLM reading companion into an educational context, this work contributes pragmatic insights to guide development of synthetic personae supporting learning. Broader impacts compel policy and industry actions to uphold responsible design in order to maximize benefits of AI integration while prioritizing student wellbeing.