Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Turning Language Model Training from Black Box into a Sandbox

Nicolas Pope, Matti Tedre

TL;DR

Problem: LLMs remain opaque, limiting learners' understanding of how data and training shape behavior. Approach: a browser-based Little Language Machine enables in-browser training of a small transformer, making the complete workflow visible from data curation to generation. Findings: a brief hands-on training shifts explanations toward data-centered reasoning; misattributions decrease while data-related explanations increase; a statistical test shows significance: $z = 5.09$, $p < .001$. Significance: supports integrating transparent, trainable AI systems into introductory computing to foster grounded AI literacy and shift from prompting to training.

Abstract

Most classroom engagements with generative AI focus on prompting pre-trained models, leaving the role of training data and model mechanics opaque. We developed a browser-based tool that allows students to train a small transformer language model entirely on their own device, making the training process visible. In a CS1 course, 162 students completed pre- and post-test explanations of why language models sometimes produce incorrect or strange output. After a brief hands-on training activity, students' explanations shifted significantly from anthropomorphic and misconceived accounts toward data- and model-based reasoning. The results suggest that enabling learners to directly observe training can support conceptual understanding of the data-driven nature of language models and model training, even within a short intervention. For K-12 AI literacy and AI education research, the study findings suggest that enabling students to train - and not only prompt - language models can shift how they think about AI.

Turning Language Model Training from Black Box into a Sandbox

TL;DR

Problem: LLMs remain opaque, limiting learners' understanding of how data and training shape behavior. Approach: a browser-based Little Language Machine enables in-browser training of a small transformer, making the complete workflow visible from data curation to generation. Findings: a brief hands-on training shifts explanations toward data-centered reasoning; misattributions decrease while data-related explanations increase; a statistical test shows significance: , . Significance: supports integrating transparent, trainable AI systems into introductory computing to foster grounded AI literacy and shift from prompting to training.

Abstract

Most classroom engagements with generative AI focus on prompting pre-trained models, leaving the role of training data and model mechanics opaque. We developed a browser-based tool that allows students to train a small transformer language model entirely on their own device, making the training process visible. In a CS1 course, 162 students completed pre- and post-test explanations of why language models sometimes produce incorrect or strange output. After a brief hands-on training activity, students' explanations shifted significantly from anthropomorphic and misconceived accounts toward data- and model-based reasoning. The results suggest that enabling learners to directly observe training can support conceptual understanding of the data-driven nature of language models and model training, even within a short intervention. For K-12 AI literacy and AI education research, the study findings suggest that enabling students to train - and not only prompt - language models can shift how they think about AI.
Paper Structure (5 sections, 2 figures)

This paper contains 5 sections, 2 figures.

Table of Contents

  1. Introduction
  2. System Description
  3. Methods
  4. Results and Discussion
  5. Conclusion

Figures (2)

  • Figure 1: Screen shot of Little Language Machine. When no XAI options are selected, the basic interface consists of five main windows: ① Training data selection, ④ Model selection, ⑤ Training process tracker, ⑥ Model quality evaluation, and ⑦ Text generator window.
  • Figure 2: Students' responses to the task on how and why language models sometimes produce text that is incorrect, implausible, or nonsensical.