Stochastic Resonance Improves the Detection of Low Contrast Images in Deep Learning Models

TL;DR

This study investigates stochastic resonance in rate-based neural networks by training a 20-unit LSTM on MNIST with an added empty stimulus class and evaluating test-time performance under sub-threshold inputs with added noise. The results show a bell-shaped dependence of accuracy on noise level, indicating SR, with optimal noise varying by stimulus strength and with both uniform and Gaussian noise yielding similar improvements. Crucially, the presence of an explicit no-stimulus class is required for SR to emerge, and SR effects persist even with a single-step sequence. The findings suggest noise-aware test-time perturbations could enhance robustness of rate-based RNNs to weak signals, and point to future work on adaptive noise and training with noise to extend SR to other architectures.

Abstract

Stochastic resonance describes the utility of noise in improving the detectability of weak signals in certain types of systems. It has been observed widely in natural and engineered settings, but its utility in image classification with rate-based neural networks has not been studied extensively. In this analysis a simple LSTM recurrent neural network is trained for digit recognition and classification. During the test phase, image contrast is reduced to a point where the model fails to recognize the presence of a stimulus. Controlled noise is added to partially recover classification performance. The results indicate the presence of stochastic resonance in rate-based recurrent neural networks.

Figures (6)

• Figure 1: Typical output performance versus noise level in systems capable of stochastic resonance. Adapted from mcdonnell2009stochastic.
• Figure 2: Uniform noise: mean and standard deviation of validation accuracy with different thresholding factors and uniform noise levels. t=1.0 corresponds to the original image.
• Figure 3: Gaussian noise: mean and standard deviation of validation accuracy with different thresholding factors and Gaussian noise levels. t=1.0 corresponds to the original image.
• Figure 4: Without no-signal class: mean and standard deviation of validation accuracy with different thresholding factors and uniform noise levels. t=1.0 corresponds to the original image.
• Figure 5: Mean and standard deviation of validation accuracy for different LSTM sequence lengths (t-factor=0.15, uniform noise level=0.075).