Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DEALing with Image Reconstruction: Deep Attentive Least Squares

Mehrsa Pourya, Erich Kobler, Michael Unser, Sebastian Neumayer

TL;DR

DEAL addresses ill-posed image reconstruction by marrying traditional regularization with deep learning through a Tikhonov-inspired, quadratic regularizer whose weights are learned and spatially adapted by an attention mechanism. The method iteratively solves a linear system $\mathbf{A}_k\mathbf{x}_{k+1}=\mathbf{b}$ with $\mathbf{A}_k=\mathbf{H}^\top\mathbf{H}+\lambda\mathbf{W}^\top\mathbf{M}(\mathbf{x}_k)^2\mathbf{W}$ using conjugate gradients, while $\mathbf{M}$ is produced by a lightweight network that modulates regularization at edges and complex structures. The authors provide theoretical guarantees: uniqueness of each update under mild kernel conditions, existence of a fixed point, Lipschitz continuity and possible contraction leading to exponential convergence, and a stability bound with respect to measurement changes. Empirically, DEAL achieves competitive results on grayscale/color denoising, color super-resolution, and MRI reconstruction with fewer parameters than many deep baselines and demonstrates strong interpretability through visualizations of the learned masks and filters. This framework offers a principled, universal, and robust alternative to plug-and-play and learned regularizers for diverse inverse problems.

Abstract

State-of-the-art image reconstruction often relies on complex, highly parameterized deep architectures. We propose an alternative: a data-driven reconstruction method inspired by the classic Tikhonov regularization. Our approach iteratively refines intermediate reconstructions by solving a sequence of quadratic problems. These updates have two key components: (i) learned filters to extract salient image features, and (ii) an attention mechanism that locally adjusts the penalty of filter responses. Our method achieves performance on par with leading plug-and-play and learned regularizer approaches while offering interpretability, robustness, and convergent behavior. In effect, we bridge traditional regularization and deep learning with a principled reconstruction approach.

DEALing with Image Reconstruction: Deep Attentive Least Squares

TL;DR

DEAL addresses ill-posed image reconstruction by marrying traditional regularization with deep learning through a Tikhonov-inspired, quadratic regularizer whose weights are learned and spatially adapted by an attention mechanism. The method iteratively solves a linear system with using conjugate gradients, while is produced by a lightweight network that modulates regularization at edges and complex structures. The authors provide theoretical guarantees: uniqueness of each update under mild kernel conditions, existence of a fixed point, Lipschitz continuity and possible contraction leading to exponential convergence, and a stability bound with respect to measurement changes. Empirically, DEAL achieves competitive results on grayscale/color denoising, color super-resolution, and MRI reconstruction with fewer parameters than many deep baselines and demonstrates strong interpretability through visualizations of the learned masks and filters. This framework offers a principled, universal, and robust alternative to plug-and-play and learned regularizers for diverse inverse problems.

Abstract

State-of-the-art image reconstruction often relies on complex, highly parameterized deep architectures. We propose an alternative: a data-driven reconstruction method inspired by the classic Tikhonov regularization. Our approach iteratively refines intermediate reconstructions by solving a sequence of quadratic problems. These updates have two key components: (i) learned filters to extract salient image features, and (ii) an attention mechanism that locally adjusts the penalty of filter responses. Our method achieves performance on par with leading plug-and-play and learned regularizer approaches while offering interpretability, robustness, and convergent behavior. In effect, we bridge traditional regularization and deep learning with a principled reconstruction approach.

Paper Structure

This paper contains 34 sections, 4 theorems, 23 equations, 15 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Contribution
  3. Related Literature
  4. Learned Regularization
  5. Spatial Adaptivity
  6. Iterative Refinement
  7. Nonlocal Laplacians
  8. Methodology
  9. Architecture
  10. Reconstruction Block
  11. Mask Generation Block
  12. Multi-Conv Block
  13. Training
  14. Dataset and Loss
  15. Gradient Tracking
  16. ...and 19 more sections

Key Result

Proposition 4.1

If $\ker ({\bf{H}}) \cap \ker ({\bf{M}}({\bf{x}}_k) {\bf{W}}) = \{{\bf{0}}\}$, then ${\bf{A}}_k$ is positive definite and eq:UpdateEquation has a unique solution. Moreover, if ${\bf{M}}^2({\bf{x}}_k) \succeq \epsilon_M \mathrm{Id}$, then ${\bf{A}}_k \succeq {\bf{H}}^\top {\bf{H}} + \epsilon_M {\bf{W

Figures (15)

  • Figure 1: DEAL generates a sequence of reconstructions ${\bf{x}}_k$ via \ref{['eq:IterRefine']} from the inputs ${\bf{y}}$ and ${\bf{H}}$, initalization ${\bf{x}}_{0} = {\bf{0}}$, and hyper-parameters $\sigma$ and $\lambda$. When the stop condition is met, it returns $\hat{{\bf{x}}}$.
  • Figure 2: Architecture of the mask generation block.
  • Figure 3: Denoising of the castle image for $\sigma = 25$. For each reconstruction (PSNR and SSIM) is provided.
  • Figure 4: Solution path and channel-wise averages $\overline{{\bf{M}}}$ of the weights for DEAL iterations, exemplified with the castle image and $\sigma_n = 25$. On the right, we plot the residual values and PSNR over the number of outer iteration $k$.
  • Figure 5: Superresolution task with rate $s=2$ and $\sigma_n=2.55$. The bottom image illustrates the dependence on initialization.
  • ...and 10 more figures

Theorems & Definitions (8)

  • Proposition 4.1
  • Lemma 4.2
  • Theorem 4.3
  • Theorem 4.4
  • proof
  • proof
  • proof
  • proof