Observing Without Doing: Pseudo-Apprenticeship Patterns in Student LLM Use

TL;DR

The paper investigates how CS1 students integrate large language models into programming tasks, revealing a prevalent pseudo-apprenticeship pattern where students accept AI-generated solutions but avoid the later stages of cognitive apprenticeship that foster autonomy. Using a mixed-methods design with 14 undergraduates, think-aloud protocols, and three tasks of increasing openness, the study characterizes when and how students prompt LLMs, how they use AI outputs, and their attitudes toward AI use. It shows that students rely on AI especially for unfamiliar or open-ended tasks, often importing complete solutions and avoiding productive struggle, which can impede independent problem-solving. The authors propose instructional interventions—designing intentional workflows, supporting early struggle, and teaching AI-use traps—to preserve learning benefits while acknowledging pervasive AI use in real-world programming practice.

Abstract

Large Language Models (LLMs) such as ChatGPT have quickly become part of student programmers' toolkits, whether allowed by instructors or not. This paper examines how introductory programming (CS1) students integrate LLMs into their problem-solving processes. We conducted a mixed-methods study with 14 undergraduates completing three programming tasks while thinking aloud and permitted to access any resources they choose. The tasks varied in open-endedness and familiarity to the participants and were followed by surveys and interviews. We find that students frequently adopt a pattern we call pseudo-apprenticeship, where students engage attentively with expert-level solutions provided by LLMs but fail to participate in the stages of cognitive apprenticeship that promote independent problem-solving. This pattern was augmented by disconnects between students' intentions, actions, and self-perceived behavior when using LLMs. We offer design and instructional interventions for promoting learning and addressing the patterns of dependent AI use observed.

Figures (8)

• Figure 1: Time Spent Per Prompt, by Participant. Participants are ordered by number of prompts.
• Figure 2: Time Spent Interacting with LLM, by Activity.
• Figure 3: Time Spent Before First LLM Use (excludes P7, Activity 2 where participant searched web before later using LLM)
• Figure 4: Information Inclusion in Prompts
• Figure 5: Flow of Prompting Intentions: From Initial to Subsequent LLM Use. Arrows represent transitions from a participant's initial prompting intention to all subsequent intentions within the same activity. Tick marks indicate individual participant-activity pairs. Loops show cases where only one intention was used throughout an activity.