Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Supervised Learning from Noisy and Incomplete Data

Julián Tachella, Mike Davies

TL;DR

This work surveys self-supervised learning methods for inverse problems where measurements are noisy or incomplete. It systematically catalogs unbiased and constrained loss designs, and how they relate to supervised losses, including connections to SURE, Noise2Noise, and NEF-based generalizations. It covers learning from multiple forward operators, leveraging transformation invariances, and learning generative models to recover or sample from $p(\boldsymbol{x}|\boldsymbol{y})$, while addressing model identifiability and finite-data effects. The manuscript also discusses practical extensions to non-linear problems, large-scale imaging, robustness, uncertainty quantification, and the sample complexity of self-supervised approaches. Overall, it provides a cohesive theoretical and practical framework for self-supervised imaging and sensing in diverse inverse-problem settings.

Abstract

Many important problems in science and engineering involve inferring a signal from noisy and/or incomplete observations, where the observation process is known. Historically, this problem has been tackled using hand-crafted regularization (e.g., sparsity, total-variation) to obtain meaningful estimates. Recent data-driven methods often offer better solutions by directly learning a solver from examples of ground-truth signals and associated observations. However, in many real-world applications, obtaining ground-truth references for training is expensive or impossible. Self-supervised learning methods offer a promising alternative by learning a solver from measurement data alone, bypassing the need for ground-truth references. This manuscript provides a comprehensive summary of different self-supervised methods for inverse problems, with a special emphasis on their theoretical underpinnings, and presents practical applications in imaging inverse problems.

Self-Supervised Learning from Noisy and Incomplete Data

TL;DR

This work surveys self-supervised learning methods for inverse problems where measurements are noisy or incomplete. It systematically catalogs unbiased and constrained loss designs, and how they relate to supervised losses, including connections to SURE, Noise2Noise, and NEF-based generalizations. It covers learning from multiple forward operators, leveraging transformation invariances, and learning generative models to recover or sample from , while addressing model identifiability and finite-data effects. The manuscript also discusses practical extensions to non-linear problems, large-scale imaging, robustness, uncertainty quantification, and the sample complexity of self-supervised approaches. Overall, it provides a cohesive theoretical and practical framework for self-supervised imaging and sensing in diverse inverse-problem settings.

Abstract

Many important problems in science and engineering involve inferring a signal from noisy and/or incomplete observations, where the observation process is known. Historically, this problem has been tackled using hand-crafted regularization (e.g., sparsity, total-variation) to obtain meaningful estimates. Recent data-driven methods often offer better solutions by directly learning a solver from examples of ground-truth signals and associated observations. However, in many real-world applications, obtaining ground-truth references for training is expensive or impossible. Self-supervised learning methods offer a promising alternative by learning a solver from measurement data alone, bypassing the need for ground-truth references. This manuscript provides a comprehensive summary of different self-supervised methods for inverse problems, with a special emphasis on their theoretical underpinnings, and presents practical applications in imaging inverse problems.
Paper Structure (79 sections, 20 theorems, 138 equations, 12 figures, 3 tables)

This paper contains 79 sections, 20 theorems, 138 equations, 12 figures, 3 tables.

Table of Contents

  1. Introduction to self-supervised learning for inverse problems
  2. Inverse problems
  3. From analytic reconstruction to machine learning
  4. Why it is hard to invert?
  5. Model based reconstruction
  6. Supervised learning
  7. Commonly used network models
  8. Limitations of Supervised Learning
  9. Self-supervised learning
  10. Unbiased losses
  11. Constrained losses
  12. Losses sharing global minimum
  13. Learning a generative model
  14. What this manuscript is not about
  15. Self-supervised representation learning
  16. ...and 64 more sections

Key Result

Proposition 2.1

Let $\boldsymbol{y}_1$ and $\boldsymbol{y}_2$ be two random variables independent conditional on $\boldsymbol{x}$, and assume that $\mathbb{E}_{\boldsymbol{y}_2|\boldsymbol{x}}\left\{\boldsymbol{y}_2\right\}=\boldsymbol{x}$, then where the constant is independent of $f$.

Figures (12)

  • Figure 1: Supervised and self-supervised learning. Supervised learning requires a dataset of paired data $\{(\boldsymbol{x}_i,\boldsymbol{y}_i)\}_{i=1}$, whereas self-supervised learning, the main focus of this manuscript, relies on measurement data alone $\{\boldsymbol{y}_i\}_{i=1}$, and consists of constructing losses that do not require ground truth data, and can approximate the supervised loss.
  • Figure 2: The expressivity-robustness trade-off in self-supervised denoising tachella_unsure_2025. As the assumptions about the noise distribution are relaxed, the learned estimator needs to be less expressive to avoid over-fitting the noise.
  • Figure 3: Self-supervised denoising across noise levels tachella_unsure_2025. Comparison of supervised, \ref{['eq:sure']}, \ref{['eq: R2R']} and \ref{['eq: unsure']} losses on an MNIST Gaussian denoising task with a U-Net denoiser. If the noise level $\sigma$ is correctly specified in SURE and R2R, the performance is close to the supervised case. However, if the noise level is misspecified, the performance drops significantly. The UNSURE loss is robust to noise level misspecification, and performs close to the supervised case.
  • Figure 4: Pixel splitting strategies. Noise2Void and Noise2Self zero-fill or copy neighboring values to the pixels removed from the input image, whereas Neighbor2Neighbor splits using two random subsamplings of every $2\times 2$ neighborhood, one as input and the other as the target.
  • Figure 5: Learning with multiple forward operators. In some imaging settings, such as image inpainting (left) and accelerated MRI (right), the masking operator changes across samples in the dataset, offering different views of the signal distribution.
  • ...and 7 more figures

Theorems & Definitions (33)

  • Proposition 2.1
  • proof
  • Proposition 2.2: Pang et al. pang_recorrupted--recorrupted_2021
  • proof
  • Theorem 2.3: Monroy et al. monroy_generalized_2025
  • Lemma 2.4: Stein stein_estimation_1981
  • Lemma 2.5: Hudson hudson_natural_1978
  • Proposition 2.6: Monroy et al. monroy_generalized_2025
  • Theorem 2.7: Tachella et al. tachella_unsure_2025
  • Proposition 2.8: Adapted from Batson and Royer batson_noise2self_2019
  • ...and 23 more