Deterministic versus stochastic dynamical classifiers: opposing random adversarial attacks with noise

TL;DR

This work reframes classification as a dynamical-system problem by training a CVFR (continuous Hopfield) network to embed a set of stable attractors via spectral decomposition of the coupling matrix, effectively sculpting the basins of attraction so that input initial conditions converge to class-specific equilibria. It introduces a stochastic CVFR variant with multiplicative noise that fades near the planted attractors, and demonstrates that this noise improves robustness to random adversarial perturbations on stylized letters and MNIST while maintaining competitive accuracy. The deterministic model achieves high accuracy (letters 99.9%; MNIST 97.2%), and the stochastic version shows enhanced resilience to attacks, highlighting the constructive interplay between noise and dynamics in dynamical classifiers. Overall, the paper provides a principled dynamical-systems perspective on learning that leverages spectral attractor planting and noise-assisted resilience for robust pattern recognition.

Abstract

The Continuous-Variable Firing Rate (CVFR) model, widely used in neuroscience to describe the intertangled dynamics of excitatory biological neurons, is here trained and tested as a veritable dynamically assisted classifier. To this end the model is supplied with a set of planted attractors which are self-consistently embedded in the inter-nodes coupling matrix, via its spectral decomposition. Learning to classify amounts to sculp the basin of attraction of the imposed equilibria, directing different items towards the corresponding destination target, which reflects the class of respective pertinence. A stochastic variant of the CVFR model is also studied and found to be robust to aversarial random attacks, which corrupt the items to be classified. This remarkable finding is one of the very many surprising effects which arise when noise and dynamical attributes are made to mutually resonate.

Figures (6)

• Figure 1: Panel (a): Schematic representation of the dynamical model employed. It is made by interacting neurons, each being subject to a linear local dynamics. The non linear activation term hides inside the arrow that points to the directed inter-nodes interactions. In the application that we shall discuss, each neuron is uniquely associated to a single pixel of the image to be analyzed. Panel (b): Schematic representation of the discrete Euler map deployed on a feedforward neural architecture.
• Figure 2: Examples of input from letters (left top panel) and MNIST dataset (left bottom panel), shown alongside with their corresponding target attractors (on the right). In principle the asymptotic attractors can be assigned any patterns by employing the triplet $(0,x_m,x_p)$. In this case we have arbitrarily chosen to employ the null element and $x_p$, with no loss of generality.
• Figure 3: Temporal evolution of the neurons' activity (i.e., firing rate) in the trained network. Panel (a) refers to a data point from the letters dataset, while panel (b) shows the evolution of a data point from the MNIST dataset. The colors of the trajectories stand for the final state that the corresponding node must reach to achieve accurate classification. In both cases, the activity at $t=0$ reflects the intensity of the pixels in the image to be classified. After an initial time window displaying a seemingly irregular evolution, the trajectories converge to their respective target values, indicating that the model has correctly classified the input datum.
• Figure 4: Multiple temporal evolutions of non linear image $f(x_i)$ of the state variable $x_i$, for a fixed node $i$, and for a given data input selected from the MNIST dataset. The depicted curves refer to different realizations of the trained stochastic model with $\sigma=1.0$. The variability of the trajectories depends solely on the stochastic nature of the - post-trained - evaluation process. As time progresses, all trajectories converge to the target fixed point, and the stochasticity fades away, as follows the damping factor (\ref{['eq:damping_factor']}). The inset shows the standard deviation of the generated trajectories, as computed numerically: at $t=0$, there is no variability (the initial data point is always the same), while at subsequent times, the standard deviation increases, before eventually converging to zero at late times.
• Figure 5: Letters dataset. The accuracy on the test set is reported as a function of the parameter $p$ that quantifies the intensity of the attack. The upper panel refers to a type A attack, while the lower panel refers to a random type B attack (see the main body of the manuscript). Different lines correspond to different models. Each model was trained with a different level of inherent noise, as quantified by the amplitude parameter $\sigma$, shown in the legend. Specifically, $\sigma = 0.0$ corresponds to the deterministic case. As expected, the models become progressively less effective as the parameter $p$ increases. However, the stochastic models show greater resistance to external noisy attacks as compared to their deterministic analogues.