Harnessing the Power of Noise: A Survey of Techniques and Applications

TL;DR

This survey reframes noise as a constructive resource rather than a nuisance, unifying stochastic resonance and noise-enhanced techniques across signal processing, image processing, machine learning, graph analysis, NLP, and privacy. It presents theoretical foundations such as the Noise Benefit Theorem and the NEM theorem, and details practical strategies for injecting noise into inputs, outputs, hidden layers, weights, and gradients to improve convergence, generalization, and robustness. The authors also cover noise-enabled applications in graph learning, NLP, and recommender systems, along with broader natural science and privacy implications via differential privacy and graph perturbation. The work highlights the practical significance of controlled noise for data augmentation, privacy protection, and optimization, and outlines promising directions for integrating noise strategies with large language models, synthetic data, and graph regularization to advance real-world systems.

Abstract

Noise, traditionally considered a nuisance in computational systems, is reconsidered for its unexpected and counter-intuitive benefits across a wide spectrum of domains, including nonlinear information processing, signal processing, image processing, machine learning, network science, and natural language processing. Through a comprehensive review of both historical and contemporary research, this survey presents a dual perspective on noise, acknowledging its potential to both disrupt and enhance performance. Particularly, we highlight how noise-enhanced training strategies can lead to models that better generalize from noisy data, positioning noise not just as a challenge to overcome but as a strategic tool for improvement. This work calls for a shift in how we perceive noise, proposing that it can be a spark for innovation and advancement in the information era.

Figures (3)

• Figure 1: Typical curve of "output performance" versus "input noise magnitude" for systems capable of stochastic resonance mcdonnell2009stochastic. For small or large levels of noise, the performance improvement is limited, while performance reaches its maximum value when there is an appropriate amount of noise.
• Figure 2: One-bit image quantization. chen2014noise. (a) The original eight-bit image. (b) One-bit uniformly quantized image. (c) One-bit uniformly quantized image with additive noise. Noise helped to preserve details like contours and textures. (d) One-bit subthreshold quantized image. (e) One-bit subthreshold quantized image with additive noise which helped to recover some visual information.
• Figure 3: Detected Communities (ovals) before (a) and after (b) adding noise (dashed edge) using the same community detection algorithm (Leading Eigenvector method). Adding noise edge $(3,5)$ in (b) helps find better communities, as observed by $30\%$ decrease in the value of community detection objective function (edge cut, here).

Theorems & Definitions (9)

• Theorem 1: Markov Chain Noise Benefit Theorem franzke2011noise
• Theorem 2
• Corollary 1: NEM Condition for Gaussian Mixture Models (NEM-GMM) osoba2013noisy
• Theorem 3: Clustering Noise Benefit Theorem osoba2013noise
• Theorem 4: Hyperplane Noise Benefit Condition for Feedforward Neural Networks audhkhasi2013noise
• Theorem 5: Sphere Noise Benefit Condition for Feedforward Neural Networks audhkhasi2013noise
• Theorem 6: Hyperplane Noise-Benefit Condition for CNNs audhkhasi2016noise).
• Theorem 8: $\epsilon$-Differential Privacy Theorem dwork2006calibrating
• Theorem 9: ($\epsilon$, $\delta$)-Differential Privacy Theorem dwork2014algorithmic