AnimatedLLM: Explaining LLMs with Interactive Visualizations

TL;DR

An interactive web application that provides step-by-step visualizations of a Transformer language model, AnimatedLLM, which runs entirely in the browser, using pre-computed traces of open LLMs applied on manually curated inputs.

Abstract

Large language models (LLMs) are becoming central to natural language processing education, yet materials showing their mechanics are sparse. We present AnimatedLLM, an interactive web application that provides step-by-step visualizations of a Transformer language model. AnimatedLLM runs entirely in the browser, using pre-computed traces of open LLMs applied on manually curated inputs. The application is available at https://animatedllm.github.io, both as a teaching aid and for self-educational purposes.

Figures (6)

• Figure 1: Detailed visualization explaining how the Transformer model is pre-trained on data. Individual components show how the tokens get processed throughout the model.
• Figure 2: Simplified text generation visualization showing the autoregressive decoding process with probability distributions over candidate tokens.
• Figure 3: Front page of the website providing access to the individual views.
• Figure 4: Simplified view of the pre-training process, focusing on comparing the predicted and target distribution.
• Figure 5: The user interface during the detailed visualization of the decoding process. The labels on the side help to understand what is happening in the current animation step.