SASNet: Spatially-Adaptive Sinusoidal Neural Networks

TL;DR

SASNet tackles spectral bias, training instability, and overfitting in sinusoidal neural networks used as implicit neural representations by introducing a fixed frequency embedding layer and spatially-adaptive masks that localize neuron contributions. It couples these with a lightweight, multi-scale hash grid to jointly optimize masks and network parameters, achieving explicit frequency control and region-specific detail capture. Empirical evaluation shows SASNet surpasses prior INRs in PSNR, SSIM, and edge fidelity, while maintaining compact parameter counts and faster convergence, particularly in high-frequency edge regions. The approach holds practical value for high-frequency signal reconstruction, super-resolution, and noise suppression, with potential extensions to time-varying signals and efficiency improvements.

Abstract

Sinusoidal neural networks (SNNs) have emerged as powerful implicit neural representations (INRs) for low-dimensional signals in computer vision and graphics. They enable high-frequency signal reconstruction and smooth manifold modeling; however, they often suffer from spectral bias, training instability, and overfitting. To address these challenges, we propose SASNet, Spatially-Adaptive SNNs that robustly enhance the capacity of compact INRs to fit detailed signals. SASNet integrates a frequency embedding layer to control frequency components and mitigate spectral bias, along with jointly optimized, spatially-adaptive masks that localize neuron influence, reducing network redundancy and improving convergence stability. Robust to hyperparameter selection, SASNet faithfully reconstructs high-frequency signals without overfitting low-frequency regions. Our experiments show that SASNet outperforms state-of-the-art INRs, achieving strong fitting accuracy, super-resolution capability, and noise suppression, without sacrificing model compactness.

Figures (7)

• Figure 1: We present SASNet, a robust INR that allows training SNNs with spatial localization overcoming challenges in frequency initialization, hyperparameter selection, and noisy reconstruction. First row compares, on a $\times$16 super-resolution, SIREN sitzmann2020implicit with different hyperparameter $\omega_0$'s and our SASNet. As illustrated in the first three images, blurriness, "ringing" around edges, and noisy background are common artifacts. Second row presents three jointly optimized spatially-adaptive masks for different frequency bands (low to high). As a toy example, last row shows fitting error maps of SIREN with different $\omega_0$'s and of SASNet under the same color encoding (up to 2% absolute difference).
• Figure 2: Overview of SASNet. A multi-scale hash grid is constructed to interpolate spatial features that are decoded to spatially-adaptive group masks (SGMs) for layers in the SNN. A mask is applied to a group of neurons via broadcasting and Hadamard product. The frequency embedding layer is initialized and frozen after the network construction stage. Our network contains $\mathcal{H}$ hidden Sine Layers following the design in SIREN sitzmann2020implicit. Training data flow and back-propagation paths are highlighted as in the legend.
• Figure 3: Neurons' contribution maps of different INRs for the image in \ref{['fig:teaser']}. From left to right, each column shows 9 selected maps from hidden layers of each INR. Zoom in to visualize details.
• Figure 4: Comparing the frequency spectra of SIREN and our SASNet before training. Two red rectangles indicate band limits of low- and high-frequency embedding neurons.
• Figure 5: PSNR curves over 20,000 training steps. For each method, the line plot represents the smoothed metric values, while the shaded background indicates the actual metric value range during training. WIRE exhibits instability and undergoes two distinct training stages. FFN, SAPE, and NeuRBF utilize unique learning rate schedulers, which differ from other methods that use a fixed learning rate. In contrast, SASNet demonstrates faster convergence, superior fitting accuracy, and more stable training.