Classifying States of the Hopfield Network with Improved Accuracy, Generalization, and Interpretability

TL;DR

This work tackles the problem of classifying Hopfield network states into learned, prototype, and spurious classes, extending beyond the traditional stability-ratio approach to include interpretable non-linear classifiers. By using energy-profile features from prototype-regime Hopfield networks and evaluating both shallow neural networks and support vector machines (linear and RBF), the authors demonstrate strong generalization across networks with varying numbers of prototypes, Bernoulli noise, and instance counts, while maintaining interpretability. Key findings show that simple models often outperform the stability ratio, with non-normalized energy profiles frequently yielding better performance, though normalization can aid cross-dataset generalization in some scenarios. The study also explores Dense Associative Memory as a classifier, highlighting interpretability trade-offs, and concludes that interpretable models can achieve high accuracy and generalization without sacrificing explainability, though cross-regime generalization remains a challenge.

Abstract

We extend the existing work on Hopfield network state classification, employing more complex models that remain interpretable, such as densely-connected feed-forward deep neural networks and support vector machines. The states of the Hopfield network can be grouped into several classes, including learned (those presented during training), spurious (stable states that were not learned), and prototype (stable states that were not learned but are representative for a subset of learned states). It is often useful to determine to what class a given state belongs to; for example to ignore spurious states when retrieving from the network. Previous research has approached the state classification task with simple linear methods, most notably the stability ratio. We deepen the research on classifying states from prototype-regime Hopfield networks, investigating how varying the factors strengthening prototypes influences the state classification task. We study the generalizability of different classification models when trained on states derived from different prototype tasks -- for example, can a network trained on a Hopfield network with 10 prototypes classify states from a network with 20 prototypes? We find that simple models often outperform the stability ratio while remaining interpretable. These models require surprisingly little training data and generalize exceptionally well to states generated by a range of Hopfield networks, even those that were trained on exceedingly different datasets.

Figures (22)

• Figure 1: Energy profiles of states from the standard conditions of the Hopfield network in both the normalized and non-normalized form. The mean energy profile is shown as a solid line, and the standard deviation of each neuron is shown as a shaded area around the mean. Note that the non-normalized energy profiles have an interpretable criterion at $E=0$, where states with any energy above zero are unstable (all learned states) and those with all energy below zero are stable (prototypes and spurious).
• Figure 2: Macro F1 score of neural networks trained on energy profiles from standard Hopfield conditions. Note the two plots use different vertical axis limits. The number of Hopfield networks used for training is shown by the color of the box plots, and layer sizes are shown alone the x-axis. Each combination of parameters (visualized as a separate box) is repeated ten times.
• Figure 3: Testing macro F1 score of models trained on non-normalized energy profiles, varying the number of prototypes. Each combination of parameters (visualized as a separate box) is repeated ten times.
• Figure 4: Sampled energy profiles of states when varying the number of prototypes learned by the Hopfield network. The mean energy profile is shown as a solid line, and the standard deviation of each neuron is shown as a shaded area around the mean.
• Figure 5: Energy profiles from Figure \ref{['Fig: Experiment Two Energy Profiles']} overlaid for easier comparison between networks with differing numbers of stored prototypes. The solid lines represent states from the ten prototype network, while dashed lines represent states from the twenty prototype network.