Learning About Learning: A Physics Path from Spin Glasses to Artificial Intelligence

TL;DR

This paper argues that the Hopfield network, rooted in spin-glass physics, provides a compact, pedagogically rich framework that unifies undergraduate statistical physics, dynamical systems, linear algebra, and computational methods. It derives the Hopfield energy from a Hebbian learning rule $W_{ij} = \frac{1}{N}\sum_{\mu} \xi_i^{(\mu)}\xi_j^{(\mu)}$, and shows how asynchronous updates drive the state toward memory attractors with a storage capacity of $P_{\rm max} \approx 0.138\,N$. By connecting Ising ferromagnet and SK spin-glass theory to memory retrieval, pattern stability, and energy landscapes, the work provides classroom-ready problems and a freely available simulator to enhance undergrad teaching. The approach demonstrates how energy-minimization dynamics underlie both traditional statistical physics and contemporary neural computation, suggesting curricular updates that integrate computation and learning theory into physics education. Practically, the Hopfield framework offers intuitive tools for solving combinatorial problems via near-optimal energy minimization and fosters physics students’ ability to engage with AI-relevant methodologies.

Abstract

The Hopfield model, originally inspired by spin-glass physics, occupies a central place at the intersection of statistical mechanics, neural networks, and modern artificial intelligence. Despite its conceptual simplicity and broad applicability -- from associative memory to near-optimal solutions of combinatorial optimization problems -- it is rarely integrated into standard undergraduate physics curricula. In this paper, we present the Hopfield model as a pedagogically rich framework that naturally unifies core topics from undergraduate statistical physics, dynamical systems, linear algebra, and computational methods. We provide a concise and illustrated theoretical introduction grounded in familiar physics concepts, analyze the model's energy function, dynamics, and pattern stability, and discuss practical aspects of simulation, including a freely available simulation code. To support instruction, we conclude with classroom-ready example problems designed to mirror research practice. By explicitly connecting fundamental physics to contemporary AI applications, this work aims to help prepare physics students to understand, apply, and critically engage with the computational tools increasingly central to research, industry, and society.

Figures (11)

• Figure 1: Ising ferromagnet free energy functional. Plots of Eq. \ref{['eq:isingenergy']} using units in which $k_B=1$. The minima of $g$ are the thermodynamic (observable) states of the system. A. Break of symmetry as $T$ decreases with fixed $J=1$: the equilibrium $m=0$ ($T>T_c=1$) splits into $m=\pm m_0$ for $T<T_c$; both are solutions to the equation of state, Eq. \ref{['eq:magnetCW']}. B. A similar break of symmetry happens for fixed $T=1$, but changing $J$ around $J_c=1/T=1$. Although $J$ is an effective interaction between spins and cannot be changed in real magnets, it also controls the thermodynamic (equilibrium) states; this is the fundamental feature explored by Hopfield.
• Figure 2: Average of the spin-glass Hamiltonian over the $J_{ij}$ disorder. The Hamiltonian in Eq. \ref{['eq:SKHamilton']} was averaged over 100 realizations of the random matrix $\mathbf{J}$ for all the $2^N$ states of an $N=10$ spins system. $J_{ij}$ was sampled from a Gaussian distribution with zero mean and unity standard deviation. Each network state $\vec{\sigma}\IfNoValueTF{n}{}{^{(n)}}$ is assigned an index $n$, and is put on the horizontal axis (left). The energy fluctuates around $\left\langle E\right\rangle=0$ according to a Gaussian distribution as $N\to\infty$ (right), motivating the random-energy models for spin glasses Derrida1980REMRuelle1987. Dashed line marks the state $n=2^9=512$: larger $n$ are flipped states.
• Figure 3: Retrieval of patterns.A. The state vector $\vec{\sigma}\IfNoValueTF{-NoValue-}{}{^{(-NoValue-)}}$ of a Hopfield network with $N=100$ neurons ("spins") represented on a lattice of lateral size $L=\sqrt{N}=10$ for illustration. B. Two memories are stored in the network using Eq. \ref{['eq:weighthopfield']}; the number in each site is the index of the corresponding neuron forming either an "H" pattern, memory $\vec{\xi}\IfNoValueTF{1}{}{^{(1)}}$, or an "X" pattern, memory $\vec{\xi}\IfNoValueTF{2}{}{^{(2)}}$. C. Starting from a random IC near "H" (left), iterating the network converges to "H" (right). D. Starting from a random IC near "X" (left), iterating the network converges to "X" (right).
• Figure 4: Overlap and Hopfield energy function.A. Overlap between all $2^{12}$ states and a single (random) memory pattern $\vec{\xi}\IfNoValueTF{-NoValue-}{}{^{(-NoValue-)}}$ for a network with $N=12$ neurons, Eq \ref{['eq:overlap']}. The microscopic states were sorted by their overlap value. The "staircase" structure reflects the discreteness of the system. Most of the states are uncorrelated with the memory, $M\IfNoValueTF{-NoValue-}{}{\!\!\left(-NoValue-\right)}\approx0$. The only states with $M\IfNoValueTF{-NoValue-}{}{\!\!\left(-NoValue-\right)}=\pm 1$ are the memory, $\vec{\sigma}\IfNoValueTF{n}{}{^{(n)}}=\vec{\xi}\IfNoValueTF{-NoValue-}{}{^{(-NoValue-)}}$, and antimemory, $\vec{\sigma}\IfNoValueTF{n}{}{^{(n)}}=-\vec{\xi}\IfNoValueTF{-NoValue-}{}{^{(-NoValue-)}}$. B. The inverted parabola, Eq. \ref{['eq:energydef']} for $P=1$, is the simplest function that can be used to impose that $M\IfNoValueTF{-NoValue-}{}{\!\!\left(-NoValue-\right)}=\pm$ must be minima of $\mathcal{H}\IfNoValueTF{-NoValue-}{}{\!\left(-NoValue-\right)}$. The arrows indicate the direction of energy decrease.
• Figure 5: Hopfield energy landscape.A. The Hopfield energy function, Eq. \ref{['eq:energyhopfield']}, for a network of $N=10$ neurons ("spins") interacting through a random matrix $\mathbf{W}$; it is exactly the same as the spin glass energy, Fig. \ref{['fig:randomenergy']}. B. Three realizations of the random interaction energy, but network states $\vec{\sigma}\IfNoValueTF{n}{}{^{(n)}}$ were grouped around the energy minima (network attractors) using Hamming distance, and then sorted according to their energy (decreasing when the state is to the left of the minimum, and increasing when the state is to the right of the minimum). Each realization results in an energy landscape with different attractors. C. Energy function with a single memory encoded using Eq. \ref{['eq:weighthopfield']}. D. Energy function with $P=1$, $P=2$ and $P=3$ encoded memories, with states sorted in the same way as in panel B. Spin-flip symmetry resulting from the definition, Eq. \ref{['eq:energydef']}, makes the antimemory also an attractor.