Trending in Disordered Systems and Neural Networks
Dynamics of neural scaling laws in random feature regression with powerlaw-distributed kernel eigenvalues
Training large neural networks exposes neural scaling laws for the generalization error, which points to a universal behavior across network architectures of learning in high dimensions. It was also shown that this effect persists in the limit of highly overparametrized networks as well as the Neural network Gaussian process limit. We here develop a principled understanding of the typical behavior of generalization in Neural Network Gaussian process regression dynamics. We derive a dynamical mean-field theory that captures the typical case learning dynamics: This allows us to unify multiple existing regimes of learning studied in the current literature, namely Bayesian inference on Gaussian processes, gradient flow with or without weight-decay, and stochastic Langevin training dynamics. Employing tools from statistical physics, the unified framework we derive in either of these cases yields an effective description of the high-dimensional microscopic behavior of networks dynamics in terms of lower dimensional order parameters. We show that collective training dynamics may be separated into the dynamics of N independent eigenmodes, those evolution equations are only coupled through collective response functions and a common statistics of an effective, independent noise. Our approach allows us to quantitatively explain the dynamics of the generalization error by linking spectral and dynamical properties of learning on data with power law spectra, including phenomena such as neural scaling laws and the effect of early stopping.
2602.23039
Feb 2026Disordered Systems and Neural Networks
Learning About Learning: A Physics Path from Spin Glasses to Artificial Intelligence
The Hopfield model, originally inspired by spin-glass physics, occupies a central place at the intersection of statistical mechanics, neural networks, and modern artificial intelligence. Despite its conceptual simplicity and broad applicability -- from associative memory to near-optimal solutions of combinatorial optimization problems -- it is rarely integrated into standard undergraduate physics curricula. In this paper, we present the Hopfield model as a pedagogically rich framework that naturally unifies core topics from undergraduate statistical physics, dynamical systems, linear algebra, and computational methods. We provide a concise and illustrated theoretical introduction grounded in familiar physics concepts, analyze the model's energy function, dynamics, and pattern stability, and discuss practical aspects of simulation, including a freely available simulation code. To support instruction, we conclude with classroom-ready example problems designed to mirror research practice. By explicitly connecting fundamental physics to contemporary AI applications, this work aims to help prepare physics students to understand, apply, and critically engage with the computational tools increasingly central to research, industry, and society.
2601.07635
Jan 2026Disordered Systems and Neural Networks
Nonequilibrium Thermodynamics of Associative Memory Continuous-Time Recurrent Neural Networks
Continuous-Time Recurrent Neural Networks (CTRNNs) have been widely used for their capacity to model complex temporal behaviour. However, their internal dynamics often remain difficult to interpret. In this paper, we propose a new class of CTRNNs based on Hopfield-like associative memories with asymmetric couplings. This model combines the expressive power of associative memories with a tractable mathematical formalism to characterize fluctuations in nonequilibrium dynamics. We show that this mathematical description allows us to directly compute the evolution of its macroscopic observables (the encoded features), as well as the instantaneous entropy and entropy dissipation of the system, thereby offering a bridge between dynamical systems descriptions of low-dimensional observables and the statistical mechanics of large nonequilibrium networks. Our results suggest that these nonequilibrium associative CTRNNs can serve as more interpretable models for complex sequence-encoding networks.
2511.11150
Nov 2025Disordered Systems and Neural Networks
