The Surprising Effectiveness of Noise Pretraining for Implicit Neural Representations

Abstract

The approximation and convergence properties of implicit neural representations (INRs) are known to be highly sensitive to parameter initialization strategies. While several data-driven initialization methods demonstrate significant improvements over standard random sampling, the reasons for their success -- specifically, whether they encode classical statistical signal priors or more complex features -- remain poorly understood. In this study, we explore this phenomenon through a series of experimental analyses leveraging noise pretraining. We pretrain INRs on diverse noise classes (e.g., Gaussian, Dead Leaves, Spectral) and measure their ability to both fit unseen signals and encode priors for an inverse imaging task (denoising). Our analyses on image and video data reveal a surprising finding: simply pretraining on unstructured noise (Uniform, Gaussian) dramatically improves signal fitting capacity compared to all other baselines. However, unstructured noise also yields poor deep image priors for denoising. In contrast, we also find that noise with the classic $1/|f^α|$ spectral structure of natural images achieves an excellent balance of signal fitting and inverse imaging capabilities, performing on par with the best data-driven initialization methods. This finding enables more efficient INR training in applications lacking sufficient prior domain-specific data. For more details, visit project page at https://kushalvyas.github.io/noisepretraining.html

Figures (17)

• Figure 1: In this study, we explore the effectiveness of initializing INR parameters with signals from noise classes with different properties. (a) Samples of a few of the signal types considered in this study: natural images, noise with the classic "$1/|f^\alpha|$" spectral signature of natural images, Dead Leaves noise, and Uniform noise. (b) We propose Snp, a simple but powerful data-driven INR parameter initialization method using noise samples leveraging the Strainervyas_strainer initialization framework. We pretrain an INR encoder with signal-specific decoder heads on noise samples from each type. (c) At test time, the pretrained encoder with a randomized decoder is fine-tuned on a new (real) signal. (d) Surprising test-time signal fitting results on CelebA-HQ: Snp initialization using Uniform noise performs dramatically better on signal fitting compared to all other methods.
• Figure 2: Early reconstruction quality at $T=200$ steps.Strainer, Snp (Spectrum and Uniform) learn high quality reconstructions in $200$ steps (PSNR$\approx40db+$), while the Siren output remains blurry. Power spectrum insets show that: Snp: Uniform matches closely with the ground truth, Strainer has a noisy high-frequency coverage, and Snp: Spectrum recovers most high frequencies while Siren lacks critical frequencies.
• Figure 3: Signal fitting results on CelebA-HQ images.Snp initialized using various noises demonstrate strong signal representation ability and convergence compared to widely known baseline methods such as Sirensitzmann2020implicit, IPC ipc, and Strainervyas_strainer. Interestingly, Snp pretrained with Gaussian and Uniform noises performed significantly better than all other approaches, converging rapidly to perfect fitting.
• Figure 4: Video fitting results: Snp improves fitting performance. We show frames from representation of two example videos with randomly initialized (vanilla) ResFields resfield, and Snp: Spectrum-initialized Resfields. In both examples, Snp-initialization results in a higher accuracy with clearer visible details. This implies Snp advantages also expand beyond images to videos.
• Figure 5: Video denoising results. The figure above shows denoised frames of sample videos with randomly initialized ResFields resfield, and Snp: Spectrum-initialized ResFields. In both examples, Snp: Spectrum outperforms by more than 2dB, with dramatically visual improvement, underscoring the effectiveness of Snp trained INRs for solving inverse problems.