Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Large Language Models (LLMs) through interactive Python activities

Eugenio Tufino

TL;DR

This study addresses the challenge of making Large Language Model concepts accessible to physics students by delivering hands-on Python-based activities in Google Colab that connect Word2Vec embeddings and GPT-2 to physics ideas. The approach leverages active learning to combine theory with practical exercises, visualizations, and discussion of biases and scaling in LLMs. Key contributions include a modular Colab-based curriculum with Word2Vec and GPT-2 activities, 2D embedding visualizations via PCA, exploration of analogies and biases, and demonstrations of how temperature and model size influence generation quality. The work demonstrates that such interactive, resource-light activities can build intuition about LLMs, foster critical evaluation of AI outputs, and offer a scalable template for physics education that can be extended to broader audiences.

Abstract

This paper presents an approach to introduce physics students to the basic concepts of Large Language Models (LLMs) using Python-based activities in Google Colab. The teaching strategy integrates active learning strategies and combines theoretical ideas with practical, physics-related examples. Students engage with key technical concepts, such as word embeddings, through hands-on exploration of the Word2Vec neural network and GPT-2 - an LLM that gained a lot of attention in 2019 for its ability to generate coherent and plausible text from simple prompts. The activities highlight how words acquire meaning and how LLMs predict subsequent tokens by simulating simplified scenarios related to physics. By focusing on Word2Vec and GPT-2, the exercises illustrate fundamental principles underlying modern LLMs, such as semantic representation and contextual prediction. Through interactive experimenting in Google Colab, students observe the relationship between model parameters (such as temperature) in GPT-2 and output behaviour, understand scaling laws relating data quantity to model performance, and gain practical insights into the predictive capabilities of LLMs. This approach allows students to begin to understand how these systems work by linking them to physics concepts - systems that will shape their academic studies, professional careers and roles in society.

Exploring Large Language Models (LLMs) through interactive Python activities

TL;DR

This study addresses the challenge of making Large Language Model concepts accessible to physics students by delivering hands-on Python-based activities in Google Colab that connect Word2Vec embeddings and GPT-2 to physics ideas. The approach leverages active learning to combine theory with practical exercises, visualizations, and discussion of biases and scaling in LLMs. Key contributions include a modular Colab-based curriculum with Word2Vec and GPT-2 activities, 2D embedding visualizations via PCA, exploration of analogies and biases, and demonstrations of how temperature and model size influence generation quality. The work demonstrates that such interactive, resource-light activities can build intuition about LLMs, foster critical evaluation of AI outputs, and offer a scalable template for physics education that can be extended to broader audiences.

Abstract

This paper presents an approach to introduce physics students to the basic concepts of Large Language Models (LLMs) using Python-based activities in Google Colab. The teaching strategy integrates active learning strategies and combines theoretical ideas with practical, physics-related examples. Students engage with key technical concepts, such as word embeddings, through hands-on exploration of the Word2Vec neural network and GPT-2 - an LLM that gained a lot of attention in 2019 for its ability to generate coherent and plausible text from simple prompts. The activities highlight how words acquire meaning and how LLMs predict subsequent tokens by simulating simplified scenarios related to physics. By focusing on Word2Vec and GPT-2, the exercises illustrate fundamental principles underlying modern LLMs, such as semantic representation and contextual prediction. Through interactive experimenting in Google Colab, students observe the relationship between model parameters (such as temperature) in GPT-2 and output behaviour, understand scaling laws relating data quantity to model performance, and gain practical insights into the predictive capabilities of LLMs. This approach allows students to begin to understand how these systems work by linking them to physics concepts - systems that will shape their academic studies, professional careers and roles in society.
Paper Structure (10 sections, 3 equations, 6 figures)

This paper contains 10 sections, 3 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Word2Vec: From words to numerical embeddings
  3. Biases from data
  4. Visualize word vectors in a 2D space
  5. Vector operations in the semantic space
  6. Training a Word2Vec model from scratch
  7. Exploring GPT-2: Text Generation and stochastic predictions
  8. GPT-2 Activity: Paragraph Generation
  9. Visualizing token probabilities and temperature Effects in GPT-2
  10. Conclusions

Figures (6)

  • Figure 1: Examples of using our custom-defined find_similar_words(word) function in Word2Vec with the words “physics” and “wave,” showing the top five results.
  • Figure 2: Examples of analogies from the JN using the custom-defined function analogy(word1, word2, word3, topn=1), both in physics-related contexts and in everyday scenarios.
  • Figure 3: 2D Visualization of Word Embeddings. Words that appear closer in the plot are semantically closer in the embedding space. Notice how the “quantum” group forms a distinct cluster, whereas “work” and “force” nearly overlap, indicating they are represented as very similar vectors in this model.
  • Figure 4: An example of a GPT-2 code cell that generates a paragraph for the prompt: "The importance of communication in modern society."
  • Figure 5: Probability distribution of the top 10 most likely next tokens with temperature = 0.4. Lower temperature results in a more concentrated probability distribution.
  • ...and 1 more figures