Impact of white noise in artificial neural networks trained for classification: performance and noise mitigation strategies

TL;DR

The paper addresses internal white Gaussian noise in hardware neural networks used for classification, focusing on additive and multiplicative noise at the neuronal level with a softmax readout. It analyzes a MNIST-based network, showing that noise in the hidden layer degrades accuracy more than output-layer noise, and demonstrates two mitigation strategies: neuron pooling to average away uncorrelated noise and ghost neurons to suppress correlated additive noise. The results quantify how much noise can be tolerated and provide design rules (e.g., $W_g = -\\sum_i W^n_{i,j}$) for implementing mitigation in hardware. The findings advance practical hardware neural network design by offering robust, non-retraining-based noise suppression methods applicable to deeper architectures.

Abstract

In recent years, the hardware implementation of neural networks, leveraging physical coupling and analog neurons has substantially increased in relevance. Such nonlinear and complex physical networks provide significant advantages in speed and energy efficiency, but are potentially susceptible to internal noise when compared to digital emulations of such networks. In this work, we consider how additive and multiplicative Gaussian white noise on the neuronal level can affect the accuracy of the network when applied for specific tasks and including a softmax function in the readout layer. We adapt several noise reduction techniques to the essential setting of classification tasks, which represent a large fraction of neural network computing. We find that these adjusted concepts are highly effective in mitigating the detrimental impact of noise.

Figures (4)

• Figure 1: Schematic representation of considered ANN (a) and process of its training (b). Panel (c) shows how the internal noise is included into one neuron.
• Figure 2: Changes in accuracy of trained ANN depending on noise intensities of four separate noise sources additive uncorrelated noise $D^U_A$ (green in (a,c)), additive correlated noise $D^C_A$ (orange in (a,c)), multiplicative uncorrelated noise $D^U_M$ (blue in (b,d)) and multiplicative correlated noise $D^C_M$ (pink in (b,d)). The noise is introduced into hidden layer in the top panels, and into the last layer in the bottom panels. Solid curves correspond to training dataset, while dashed curves were prepared for testing dataset.
• Figure 3: Application of pooling technique to ANN with noisy neurons in hidden layer. Panel (a) shows the scheme of pooling technique with $m=3$. Panels (b) and (c) shows how the dependency of final accuracy on intensity of uncorrelated additive (b) and multiplicative (c) noise. In both panels, solid dark curves correspond to original case without pooling, while the remaining curves correspond to the results of using pooling technique with $m=3$, $5$, $10$ and $20$ (larger $m$ -- lighter color). Panel (d) shows how does the minimum accuracy change depending on $m$ for uncorrelated additive (green) and multiplicative (blue) noise.
• Figure 4: Application of ghost neuron technique to ANN with noisy neurons in hidden layer. Panel (a) shows the scheme of ghost neuron technique. Panels (b) and (c) shows how the impact of additive correlated noise can be changed using ghost neurons of all three types. Panel (b) and (c) were prepared for training and testing datasets, respectively.