Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident

TL;DR

This work addresses whether large language models can faithfully simulate novice think-aloud reasoning and metacognitive judgments in stepwise chemistry problems. It compares GPT-4.1 continuations under simple and extended context to human think-aloud data across 630 step-level interactions, using cosine similarity for reasoning alignment and calibration metrics for metacognition. Key findings show that while LLMs produce fluent, expert-like narratives, they are over-coherent and less variable than novices, with extended context amplifying this misalignment; their step-level performance predictions are systematically overestimated and only weakly discriminative. The study highlights epistemic limits of using LLMs as learner models and provides an evaluation framework to guide the development of adaptive tutoring systems grounded in actual learner data and constraints.

Abstract

Large language models (LLMs) are increasingly embedded in AI-based tutoring systems. Can they faithfully model novice reasoning and metacognitive judgments? Existing evaluations emphasize problem-solving accuracy, overlooking the fragmented and imperfect reasoning that characterizes human learning. We evaluate LLMs as novices using 630 think-aloud utterances from multi-step chemistry tutoring problems with problem-solving logs of student hint use, attempts, and problem context. We compare LLM-generated reasoning to human learner utterances under minimal and extended contextual prompting, and assess the models' ability to predict step-level learner success. Although GPT-4.1 generates fluent and contextually appropriate continuations, its reasoning is systematically over-coherent, verbose, and less variable than human think-alouds. These effects intensify with a richer problem-solving context during prompting. Learner performance was consistently overestimated. These findings highlight epistemic limitations of simulating learning with LLMs. We attribute these limitations to LLM training data, including expert-like solutions devoid of expressions of affect and working memory constraints during problem solving. Our evaluation framework can guide future design of adaptive systems that more faithfully support novice learning and self-regulation using generative artificial intelligence.

Figures (1)

• Figure 1: Average linguistic properties of ground-truth learner utterances and model-generated reasoning under simple and complex prompting.