Self-diffusion for Solving Inverse Problems

TL;DR

This work introduces self-diffusion, a training-free framework for solving ill-posed inverse problems by iteratively adding noise to an estimate and training a single untrained denoiser to minimize data fidelity. By exploiting the spectral bias of neural networks under a scheduled noise regime, SDI achieves coarse-to-fine reconstructions without pretrained models, and demonstrates competitive results across MRI reconstruction, radar angle estimation, and low-level vision tasks. The approach is supported by theoretical insights into data fidelity versus Jacobian regularization, and is validated through simulations and multiple real-world applications, with open-source code. Overall, SDI offers a data-efficient, robust alternative to diffusion priors that broadens the applicability of learning-based inversion to settings with limited data or domain shifts.

Abstract

We propose self-diffusion, a novel framework for solving inverse problems without relying on pretrained generative models. Traditional diffusion-based approaches require training a model on a clean dataset to learn to reverse the forward noising process. This model is then used to sample clean solutions -- corresponding to posterior sampling from a Bayesian perspective -- that are consistent with the observed data under a specific task. In contrast, self-diffusion introduces a self-contained iterative process that alternates between noising and denoising steps to progressively refine its estimate of the solution. At each step of self-diffusion, noise is added to the current estimate, and a self-denoiser, which is a single untrained convolutional network randomly initialized from scratch, is continuously trained for certain iterations via a data fidelity loss to predict the solution from the noisy estimate. Essentially, self-diffusion exploits the spectral bias of neural networks and modulates it through a scheduled noise process. Without relying on pretrained score functions or external denoisers, this approach still remains adaptive to arbitrary forward operators and noisy observations, making it highly flexible and broadly applicable. We demonstrate the effectiveness of our approach on a variety of linear inverse problems, showing that self-diffusion achieves competitive or superior performance compared to other methods.

Figures (15)

• Figure 1: Demonstration of self-diffusion applied to various inverse problems across natural and medical imaging domains. (a) Image inpainting (b) Motion deblurring (c) 4× image super-resolution (SR) (d) 2D MRI reconstruction from 6$\times$ undersampled data (e) 3D MRI reconstruction from 5.7$\times$ undersampled data In each set, the leftmost image shows the degraded input, the middle shows the self-diffusion result, and the rightmost shows the ground-truth or reference.
• Figure 2: Overview of the self-diffusion framework. At each diffusion step $t$, Gaussian noise is added to the current estimate $\mathbf{x}^{\text{true}}_t$, producing a noisy input. A single self-denoiser (initialized with an untrained network) is then trained to produce $\mathbf{x}^{\text{true}}_{t-1}$ by minimizing the task-specific data fidelity loss. This iterative alternation of noising and denoising from $\mathbf{x}_T$ to $\mathbf{x}_1$ yields a coarse-to-fine reconstruction process without the need for pretrained models or external supervision.
• Figure 3: Recovery of a sparse signal composed of multiple sine waves from compressed measurements. The top plot illustrates time-domain reconstruction, comparing self-diffusion inference (SDI) to ADMM-basis pursuit (ADMM-BP) and Deep Image Prior (DIP). The bottom plot displays the frequency spectra, where SDI more accurately captures both low- and high-frequency components compared to ADMM-BP and DIP.
• Figure 4: Reconstructions and error maps to GT from 4$\times$ undersampled k-space w/o ACS lines. $\mathcal{A}^H \mathbf{y}$ is the degraded.
• Figure 5: The camera column shows the visual context captured by the camera, depicting the road, vehicles, and surroundings. The radar measurement column displays the low-resolution radar range-azimuth heatmaps $\mathbf{y}$. The recovery column presents the recovered high-resolution angle estimations.