Towards Architecture-Agnostic Untrained Network Priors for Image Reconstruction with Frequency Regularization

TL;DR

This work tackles the practical fragility of untrained network priors for image reconstruction by introducing architecture-agnostic frequency regulation. It combines three mechanisms—bandwidth-constrained input, bandwidth-controllable upsampling via a Kaiser window, and Lipschitz-regularized convolutional layers—to steer the spectral bias without architectural changes. Across MRI reconstruction and natural image tasks, the approach reduces overfitting, narrows performance gaps between architectures, and delivers significant runtime improvements while preserving accuracy, even in out-of-domain settings. The method enables compact models to rival or surpass larger networks in a zero-shot, self-supervised framework, with code made publicly available.

Abstract

Untrained networks inspired by deep image priors have shown promising capabilities in recovering high-quality images from noisy or partial measurements without requiring training sets. Their success is widely attributed to implicit regularization due to the spectral bias of suitable network architectures. However, the application of such network-based priors often entails superfluous architectural decisions, risks of overfitting, and lengthy optimization processes, all of which hinder their practicality. To address these challenges, we propose efficient architecture-agnostic techniques to directly modulate the spectral bias of network priors: 1) bandwidth-constrained input, 2) bandwidth-controllable upsamplers, and 3) Lipschitz-regularized convolutional layers. We show that, with just a few lines of code, we can reduce overfitting in underperforming architectures and close performance gaps with high-performing counterparts, minimizing the need for extensive architecture tuning. This makes it possible to employ a more compact model to achieve performance similar or superior to larger models while reducing runtime. Demonstrated on inpainting-like MRI reconstruction task, our results signify for the first time that architectural biases, overfitting, and runtime issues of untrained network priors can be simultaneously addressed without architectural modifications. Our code is publicly available.

Figures (11)

• Figure 1: Improving underperforming architectures (SSIM ($\uparrow$)). Turning the left to the right simply by replacing the white-noise input with selected Fourier features or low-pass filtering the noise input, which can be implemented with a few lines of code.
• Figure 2: Overview of the proposed regularized network priors for MRI reconstruction. By adjusting the bandwidth of the input, the interpolation-based upsampling, and regularizing the convolutional layers, our approach enables more direct control over the spectral bias of architectures with various depths and widths.
• Figure 3: (a) Visualization of the 1D white noise, the low-passed noise via Gaussian blur, and Fourier features of various frequencies in the frequency domain. (b) Limiting the input's bandwidth via either Gaussian blur or Fourier features with lower $f_c$ ($L=4$ or $8$) are both effective in alleviating overfitting and enhancing the peak performance.
• Figure 4: Narrowing input's bandwidth promotes low frequencies and enhances extrapolation capabilities as a "free lunch". The output becomes smoother as $\sigma$ increases, up to a certain point.
• Figure 5: Our approach significantly minimizes the performance gaps among architectures with various depths $\{2,5,8\}$ and widths $\{64, 256\}$.