Exploring Student-AI Interactions in Vibe Coding

TL;DR

This study investigates how introductory and advanced CS students interact with a vibe-coding platform (Replit) to build a web app, focusing on think-aloud sessions and qualitative coding of student-AI interactions. Using a two-hour, think-aloud procedure with 19 participants (CS1 and SWE) and a detailed labeling scheme, the authors identify two main patterns: (1) most actions are prototype testing and debugging prompts with limited engagement with underlying code, and (2) advanced students provide richer, code-aware context in prompts and engage more with code/log content. The findings contrast novice and experienced programmers, showing that experience influences prompt quality and willingness to inspect and modify AI-generated code, with SWE students more likely to perform code interpretation and targeted code prompts. The results have instructional implications, highlighting the need to teach both prompting strategies and fundamental programming concepts, including unit testing and code comprehension, to prepare students for AI-augmented software development. Overall, the work contributes an empirical lens on how vibe coding is adopted by students and offers guidance for educators to balance AI-assisted workflows with foundational coding skills.

Abstract

Background and Context. Chat-based and inline-coding-based GenAI has already had substantial impact on the CS Education community. The recent introduction of ``vibe coding'' may further transform how students program, as it introduces a new way for students to create software projects with minimal oversight. Objectives. The purpose of this study is to understand how students in introductory programming and advanced software engineering classes interact with a vibe coding platform (Replit) when creating software and how the interactions differ by programming background. Methods. Interview participants were asked to think-aloud while building a web application using Replit. Thematic analysis was then used to analyze the video recordings with an emphasis on the interactions between the student and Replit. Findings. For both groups, the majority of student interactions with Replit were to test or debug the prototype and only rarely did students visit code. Prompts by advanced software engineering students were much more likely to include relevant app feature and codebase contexts than those by introductory programming students.

Figures (4)

• Figure 1: Screenshot of the Replit user interface during a task.
• Figure 2: Distribution of labels across all 19 students, grouped by course. Each bar represents a single student, segmented by interaction categories: Interacting with Prototype, Writing a Prompt, Engaging with Code/log, and Managing Replit Workflow. SWE_1 to SWE_10 are SWE students and CS1_1 to CS1_9 are CS1 students.
• Figure 3: Proportion of Engaging with Code/log labels per student, normalized by each student’s total behavior labels.
• Figure 4: Proportion of Low Context prompts per student, normalized by each student’s total number of prompts.