Learning to Live with AI: How Students Develop AI Literacy Through Naturalistic ChatGPT Interaction

TL;DR

This work investigates how undergraduates develop AI literacy through naturalistic interactions with ChatGPT, revealing that competence arises from ongoing relational practice rather than one-off adoption. Using a mixed-methods pipeline, the study identifies five use-genres—academic workhorse, repair and negotiation, emotional companion, metacognitive partner, and trust calibration—and introduces repair literacy as a crucial, emergent skill. The findings show students curate genre portfolios, perform boundary testing, and engage in co-regulation with AI, extending existing AI-literacy frameworks to emphasize affective, epistemic, and social dimensions. The results offer empirical guidance for AI literacy pedagogy and responsible AI integration, emphasizing design of learning environments that support student agency and nuanced human-AI collaboration.

Abstract

How do students develop AI literacy through everyday practice rather than formal instruction? While normative AI literacy frameworks proliferate, empirical understanding of how students actually learn to work with generative AI remains limited. This study analyzes 10,536 ChatGPT messages from 36 undergraduates over one academic year, revealing five use genres -- academic workhorse, emotional companion, metacognitive partner, repair and negotiation, and trust calibration -- that constitute distinct configurations of student-AI learning. Drawing on domestication theory and emerging frameworks for AI literacy, we demonstrate that functional AI competence emerges through ongoing relational negotiation rather than one-time adoption. Students develop sophisticated genre portfolios, strategically matching interaction patterns to learning needs while exercising critical judgment about AI limitations. Notably, repair work during AI breakdowns produces substantial learning about AI capabilities, developing what we term "repair literacy" -- a crucial but underexplored dimension of AI competence. Our findings offer educators empirically grounded insights into how students actually learn to work with generative AI, with implications for AI literacy pedagogy, responsible AI integration, and the design of AI-enabled learning environments that support student agency.

Figures (7)

• Figure 1: Mixed-methods pipeline integrating human qualitative analysis with computational scaling. Grounded theory coding of 1,882 messages produced a codebook (5 categories, 41 subcodes), which guided GPT-4o annotation of 10,536 messages. Human validation confirmed substantial to near-perfect agreement ($\kappa$ = 0.75–0.91), enabling interpretive analysis that identified five emergent use genres.
• Figure 2: Timeseries of conversations. Activity dropped during spring break, summer, and winter holidays, reflecting academic use.
• Figure 3: Subcategories within Information Seeking prompts, including concept explanation, theory application, and clarification of instructions.
• Figure 4: Subcategories within Content Generation, such as code writing, job application content, multiple choice solving, and summarization.
• Figure 5: Prompt types involving Language Refinement, including grammar correction, rewording, rhetorical suggestions, and translation.