Investigating Youth's Technical and Ethical Understanding of Generative Language Models When Engaging in Construction and Deconstruction Activities

TL;DR

This work addresses the gap in youth AI literacy by examining how teenagers can develop technical and ethical understandings of generative language models through construction and deconstruction activities. It adopts an in-pieces framework to analyze how knowledge fragments related to GLMs are activated and evolve in context, using data-driven construction with datasets of 70–350K tokens trained via nanoGPT and algorithm auditing of ChatGPT. The study employs thematic analysis, explanatory case studies, clinical interviews, microgenetic knowledge analysis, and ordered network analysis to link in-the-moment conceptions with prior experience and identity, pre- and post-workshop. By providing empirical evidence and a pratiques inventory, the work aims to inform youth-centered educational practices and contribute to responsible GLM literacy development for diverse learners.

Abstract

The widespread adoption of generative artificial intelligence/machine learning (AI/ML) technologies has increased the need to support youth in developing AI/ML literacies. However, most work has centered on preparing young people to use these systems, with less attention to how they can participate in designing and evaluating them. This study investigates how engaging young people in the design and auditing of generative language models (GLMs) may foster the development of their understanding of how these systems work from both technical and ethical perspectives. The study takes an in-pieces approach to investigate novices' conceptions of GLMs. Such an approach supports the analysis of how technical and ethical conceptions evolve and relate to each other. I am currently conducting a series of participatory design workshops with sixteen ninth graders (ages 14-15) in which they will (a) build GLMs from a data-driven perspective that glassboxes how data shapes model performance and (b) audit commercial GLMs by repeatedly and systematically querying them to draw inferences about their behaviors. I will analyze participants' interactions to identify ethical and technical conceptions they may exhibit while designing and auditing GLMs. I will also conduct clinical interviews and use microgenetic knowledge analysis and ordered network analysis to investigate how participants' ethical and technical conceptions of GLMs relate to each other and change after the workshop. The study will contribute (a) evidence of how engaging youth in design and auditing activities may support the development of ethical and technical understanding of GLMs and (b) an inventory of novice design and auditing practices that may support youth's technical and ethical understanding of GLMs.