Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An analytic theory of convolutional neural network inverse problems solvers

Minh Hai Nguyen, Quoc Bao Do, Edouard Pauwels, Pierre Weiss

TL;DR

This work provides a principled analytic lens to CNN-based inverse problem solvers by casting them as constrained MMSE estimators that enforce translation equivariance and locality. It derives LE-MMSE, a closed-form estimator that accounts for finite receptive fields and forward operators, and shows how the pre-inverse B shapes signal fidelity and noise robustness. Across denoising, inpainting, and deconvolution on FFHQ, CIFAR-10, and FashionMNIST with architectures like UNet, ResNet, and PatchMLP, LE-MMSE closely predicts neural network outputs (PSNRs near or above 25 dB), revealing the relationship between learned solvers and patch-based estimators. The framework clarifies memorization versus generalization, explains OOD behavior, and offers guidance on physics-aware versus physics-agnostic designs, providing a theoretical bridge between CNNs and MMSE theory for practical reconstruction guarantees and stability analyses.

Abstract

Supervised convolutional neural networks (CNNs) are widely used to solve imaging inverse problems, achieving state-of-the-art performance in numerous applications. However, despite their empirical success, these methods are poorly understood from a theoretical perspective and often treated as black boxes. To bridge this gap, we analyze trained neural networks through the lens of the Minimum Mean Square Error (MMSE) estimator, incorporating functional constraints that capture two fundamental inductive biases of CNNs: translation equivariance and locality via finite receptive fields. Under the empirical training distribution, we derive an analytic, interpretable, and tractable formula for this constrained variant, termed Local-Equivariant MMSE (LE-MMSE). Through extensive numerical experiments across various inverse problems (denoising, inpainting, deconvolution), datasets (FFHQ, CIFAR-10, FashionMNIST), and architectures (U-Net, ResNet, PatchMLP), we demonstrate that our theory matches the neural networks outputs (PSNR $\gtrsim25$dB). Furthermore, we provide insights into the differences between \emph{physics-aware} and \emph{physics-agnostic} estimators, the impact of high-density regions in the training (patch) distribution, and the influence of other factors (dataset size, patch size, etc).

An analytic theory of convolutional neural network inverse problems solvers

TL;DR

This work provides a principled analytic lens to CNN-based inverse problem solvers by casting them as constrained MMSE estimators that enforce translation equivariance and locality. It derives LE-MMSE, a closed-form estimator that accounts for finite receptive fields and forward operators, and shows how the pre-inverse B shapes signal fidelity and noise robustness. Across denoising, inpainting, and deconvolution on FFHQ, CIFAR-10, and FashionMNIST with architectures like UNet, ResNet, and PatchMLP, LE-MMSE closely predicts neural network outputs (PSNRs near or above 25 dB), revealing the relationship between learned solvers and patch-based estimators. The framework clarifies memorization versus generalization, explains OOD behavior, and offers guidance on physics-aware versus physics-agnostic designs, providing a theoretical bridge between CNNs and MMSE theory for practical reconstruction guarantees and stability analyses.

Abstract

Supervised convolutional neural networks (CNNs) are widely used to solve imaging inverse problems, achieving state-of-the-art performance in numerous applications. However, despite their empirical success, these methods are poorly understood from a theoretical perspective and often treated as black boxes. To bridge this gap, we analyze trained neural networks through the lens of the Minimum Mean Square Error (MMSE) estimator, incorporating functional constraints that capture two fundamental inductive biases of CNNs: translation equivariance and locality via finite receptive fields. Under the empirical training distribution, we derive an analytic, interpretable, and tractable formula for this constrained variant, termed Local-Equivariant MMSE (LE-MMSE). Through extensive numerical experiments across various inverse problems (denoising, inpainting, deconvolution), datasets (FFHQ, CIFAR-10, FashionMNIST), and architectures (U-Net, ResNet, PatchMLP), we demonstrate that our theory matches the neural networks outputs (PSNR dB). Furthermore, we provide insights into the differences between \emph{physics-aware} and \emph{physics-agnostic} estimators, the impact of high-density regions in the training (patch) distribution, and the influence of other factors (dataset size, patch size, etc).
Paper Structure (66 sections, 16 theorems, 89 equations, 36 figures, 4 tables)

This paper contains 66 sections, 16 theorems, 89 equations, 36 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Empirical MMSE estimators
  3. Related works
  4. Contributions
  5. Unconstrained MMSE estimator
  6. Physics-aware estimator
  7. Memorization of the empirical MMSE
  8. Constrained MMSE estimators
  9. Functional constraints: equivariance and locality
  10. Preliminary remarks
  11. Constraints and projection
  12. Architecture equivariance and data augmentation
  13. Translation Equivariant MMSE estimator
  14. Locality and Translation Equivariance
  15. Physics-agnostic and physics-aware estimators
  16. ...and 51 more sections

Key Result

Proposition 2.1

The MMSE estimator in def:mmse writes, for any $y \in \mathbb{R}^M$, where $w(x \vert y) = \frac{\mathcal{N}\left( {B y; B A x, \sigma^2 B B^{\top}} \right)}{\sum_{x'\in \mathcal{D}} \mathcal{N}\left( {B y; B A x', \sigma^2 B B^{\top}} \right)}$.

Figures (36)

  • Figure 1: Our analytic theory accurately predicts neural network outputs across settings. We consider three inverse problems: denoising, inpainting, and deconvolution (left to right), on FFHQ, CIFAR10, and FashionMNIST datasets (top to bottom) with varying noise levels $\sigma$ (columns). For each setting, we show the measurements (top row), our analytic LE-MMSE estimator (second row), and outputs of trained UNet, ResNet, and PatchMLP models (last three rows). Theory closely matches network outputs.
  • Figure 2: The MMSE and E-MMSE estimators memorize (yield the nearest neighbor), while the LE-MMSE estimator recombines training patches to give good reconstruction.
  • Figure 3: Architectural equivariance does not always guarantee reconstruction equivariance (\ref{['eq:reconstruction_equivariance']}). Input $y$ in 2nd row is shifted. The E-MMSE estimator $\hat{x}_{\mathcal{T}}$ is reconstruction equivariant for deconvolution but not inpainting. The LE-MMSE estimator $\hat{x}_{\mathcal{T}, \text{loc}}$ is reconstruction equivariant for deconvolution and shows reduced sensitivity to shifts for inpainting.
  • Figure 4: Physics-aware ($B\!=\!A^+$, bottom) has lower variance for inpainting (left), while physics-agnostic ($B\!=\!\mathrm{I}$, top) has lower variance for deconvolution (right). Mean and pixel-wise variance are computed w.r.t $50$ noise realizations.
  • Figure 5: The green curves reveals a strong agreement (PSNR $\gtrsim\!25$ dB) between the trained UNet2D and the analytical formula for all inverse problems. Median and interquartile range (IQR) using $50$ images per $\sigma$, $P = 5\times 5$ and $B\!=\!\mathrm{I}$. See \ref{['fig:neural_vs_analytical_resnet_patch_11', 'fig:neural_vs_analytical_patchmlp_patch_11']} for other architectures.
  • ...and 31 more figures

Theorems & Definitions (43)

  • Definition 1.1: Constrained empirical MMSE estimators
  • Definition 1.2: MMSE, E-MMSE, LE-MMSE
  • Proposition 2.1: Closed-form of the MMSE
  • Remark 2.2
  • Definition 3.1: Translation equivariant
  • Definition 3.2: Local and translation equivariant
  • Proposition 3.3
  • Definition 3.4: Data-augmented MMSE estimator
  • Theorem 3.5: Closed-form of E-MMSE
  • Remark 3.6
  • ...and 33 more