Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explaining the Machine Learning Solution of the Ising Model

Roberto C. Alamino

TL;DR

The paper tackles the problem of making ML solutions in the Ising model interpretable by physics. It combines unsupervised PCA to identify the magnetization as the order parameter and temperature as the control parameter with a symmetry-informed single-layer neural network to explain how the critical temperature is inferred, showing that a logistic-regression–like classifier can implement the solution. Key findings include a decomposition of PCA variance into temperature-driven and intra-temperature components that highlights the role of $T$ as the control parameter, and a minimal NN that accurately estimates $T_c$ across multiple lattices, with an extension to a two-unit hidden layer clarifying scenarios where magnetization signs are not restricted. This work demonstrates a physics-informed path to explainable ML, enabling extraction of underlying physical principles from learned models and guiding extensions to more complex phase diagrams.

Abstract

As powerful as machine learning (ML) techniques are in solving problems involving data with large dimensionality, explaining the results from the fitted parameters remains a challenging task of utmost importance, especially in physics applications. This work shows how this can be accomplished for the ferromagnetic Ising model, the main target of several ML studies in statistical physics. Here it is demonstrated that the successful unsupervised identification of the phases and order parameter by principal component analysis, a common method in those studies, detects that the magnetization per spin has its greatest variation with the temperature, the actual control parameter of the phase transition. Then, by using a neural network (NN) without hidden layers (the simplest possible) and informed by the symmetry of the Hamiltonian, an explanation is provided for the strategy used in finding the supervised learning solution for the critical temperature of the model's continuous phase transition. This allows the prediction of the minimal extension of the NN to solve the problem when the symmetry is not known, which becomes also explainable. These results pave the way to a physics-informed explainable generalized framework, enabling the extraction of physical laws and principles from the parameters of the models.

Explaining the Machine Learning Solution of the Ising Model

TL;DR

The paper tackles the problem of making ML solutions in the Ising model interpretable by physics. It combines unsupervised PCA to identify the magnetization as the order parameter and temperature as the control parameter with a symmetry-informed single-layer neural network to explain how the critical temperature is inferred, showing that a logistic-regression–like classifier can implement the solution. Key findings include a decomposition of PCA variance into temperature-driven and intra-temperature components that highlights the role of as the control parameter, and a minimal NN that accurately estimates across multiple lattices, with an extension to a two-unit hidden layer clarifying scenarios where magnetization signs are not restricted. This work demonstrates a physics-informed path to explainable ML, enabling extraction of underlying physical principles from learned models and guiding extensions to more complex phase diagrams.

Abstract

As powerful as machine learning (ML) techniques are in solving problems involving data with large dimensionality, explaining the results from the fitted parameters remains a challenging task of utmost importance, especially in physics applications. This work shows how this can be accomplished for the ferromagnetic Ising model, the main target of several ML studies in statistical physics. Here it is demonstrated that the successful unsupervised identification of the phases and order parameter by principal component analysis, a common method in those studies, detects that the magnetization per spin has its greatest variation with the temperature, the actual control parameter of the phase transition. Then, by using a neural network (NN) without hidden layers (the simplest possible) and informed by the symmetry of the Hamiltonian, an explanation is provided for the strategy used in finding the supervised learning solution for the critical temperature of the model's continuous phase transition. This allows the prediction of the minimal extension of the NN to solve the problem when the symmetry is not known, which becomes also explainable. These results pave the way to a physics-informed explainable generalized framework, enabling the extraction of physical laws and principles from the parameters of the models.
Paper Structure (4 sections, 10 equations, 3 figures, 1 table)

This paper contains 4 sections, 10 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Unsupervised Phase Detection
  3. Supervised Detection of the Critical Temperature
  4. Conclusions

Figures (3)

  • Figure 1: (a) Average probability of being in the ferromagnetic phase averaged over 20 test sets (circles) together with the magnetization per spin (orange dashed line) from Onsager's solution. The horizontal green dashed line marks the decision boundary. (b) Scatter plot showing the almost perfect positive correlation between the magnetization of the spin configurations $s$ and the scaled value of the products $w\cdot s$ for one of the test sets from (a).
  • Figure 2: Upper plot: argument of the sigmoid for each configuration in the test set; lower plot: probability of the configuration belonging to the ferromagnetic phase.
  • Figure 3: (a) Same plot as in figure \ref{['figure:psig']}a, but now for the 1HLNN with two hidden units and trained without restrictions on the sign of the magnetization. (b) Probabilities inferred by the units from the hidden layer, named Unit 1 (inverted triangles) and Unit 2 (circles), together with the final probability for the input configuration to be in the ferromagnetic case (thick line) plotted against the magnetization values of the configurations. The plot shows the average over 500 random configurations for each value of the magnetization.