Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learned iterative networks: An operator learning perspective

Andreas Hauptmann, Ozan Öktem

TL;DR

This work presents a unified operator-learning perspective for learned iterative networks in inverse problems, separating the learned reconstruction operator from the learning problem to reveal structural commonalities across gradient-based, proximal, variational, and primal–dual unrolled methods. It argues for structured learning—through symmetries, simulator-informed design, and multi-scale architectures—to enable robust, scalable solutions to linear and nonlinear inverse problems. The chapter surveys neural-operator architectures (convolutional operators, transformers, FNOs), discusses training paradigms (supervised, self-supervised, unpaired, greedy), and details a spectrum of learned iterative schemes, including learned gradient, proximal, variational, and primal–dual networks, plus higher-order Newton-type methods. Numerical examples illustrate that, in linear settings, different update directions yield similar performance, whereas in nonlinear problems the choice of update direction substantially affects results. Overall, the work provides a cohesive framework to design and analyze learned reconstructions, highlighting practical considerations around training, domain adaptation, and computational feasibility.

Abstract

Learned image reconstruction has become a pillar in computational imaging and inverse problems. Among the most successful approaches are learned iterative networks, which are formulated by unrolling classical iterative optimisation algorithms for solving variational problems. While the underlying algorithm is usually formulated in the functional analytic setting, learned approaches are often viewed as purely discrete. In this chapter we present a unified operator view for learned iterative networks. Specifically, we formulate a learned reconstruction operator, defining how to compute, and separately the learning problem, which defines what to compute. In this setting we present common approaches and show that many approaches are closely related in their core. We review linear as well as nonlinear inverse problems in this framework and present a short numerical study to conclude.

Learned iterative networks: An operator learning perspective

TL;DR

This work presents a unified operator-learning perspective for learned iterative networks in inverse problems, separating the learned reconstruction operator from the learning problem to reveal structural commonalities across gradient-based, proximal, variational, and primal–dual unrolled methods. It argues for structured learning—through symmetries, simulator-informed design, and multi-scale architectures—to enable robust, scalable solutions to linear and nonlinear inverse problems. The chapter surveys neural-operator architectures (convolutional operators, transformers, FNOs), discusses training paradigms (supervised, self-supervised, unpaired, greedy), and details a spectrum of learned iterative schemes, including learned gradient, proximal, variational, and primal–dual networks, plus higher-order Newton-type methods. Numerical examples illustrate that, in linear settings, different update directions yield similar performance, whereas in nonlinear problems the choice of update direction substantially affects results. Overall, the work provides a cohesive framework to design and analyze learned reconstructions, highlighting practical considerations around training, domain adaptation, and computational feasibility.

Abstract

Learned image reconstruction has become a pillar in computational imaging and inverse problems. Among the most successful approaches are learned iterative networks, which are formulated by unrolling classical iterative optimisation algorithms for solving variational problems. While the underlying algorithm is usually formulated in the functional analytic setting, learned approaches are often viewed as purely discrete. In this chapter we present a unified operator view for learned iterative networks. Specifically, we formulate a learned reconstruction operator, defining how to compute, and separately the learning problem, which defines what to compute. In this setting we present common approaches and show that many approaches are closely related in their core. We review linear as well as nonlinear inverse problems in this framework and present a short numerical study to conclude.

Paper Structure

This paper contains 63 sections, 90 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Overview
  3. Operator learning
  4. Inverse problem solvers (learned reconstruction)
  5. Optimisation solvers (learning-to-optimise)
  6. Structured learning
  7. Group symmetries and equivariances
  8. Simulator informed architectures
  9. Multi-scale architectures
  10. Learning problems
  11. Supervised operator learning
  12. Unsupervised operator learning
  13. Fully unsupervised and self-supervised training data
  14. Ground truth examples as training data
  15. Unpaired training data
  16. ...and 48 more sections

Figures (3)

  • Figure 1: Illustration of measurement setup. (Left) ground-truth, (Middle) sinogram with 120 angles and 2% noise, (Right) Reconstruction via filtered backprojection.
  • Figure 2: Comparison of reconstructions for the trained networks for one sample in the test data. From left to right and top to bottom: Ground truth, Variational Network, Proximal Network, Learned Least Squares, Learned Primal-Dual (5 iterates), Learned Primal-Dual (10 iterates).
  • Figure 3: Comparison of training update directions and training schemes for learned reconstructions in QPAT. Shown is the relative error in reconstructed absorption coefficients $\mu_a$ for (Left) greedy training schemes and (Right) end-to-end, with three choices for the handcrafted update direction $\triangle\!f$ as gradient descent (GD), Gauss-Newton (GN), and quasi-Newton SR1 update (SR1).

Theorems & Definitions (8)

  • Definition 2.1: Learned reconstruction operator
  • Remark 3.1
  • Remark 4.1
  • Remark 4.2
  • Remark 5.1
  • Remark 6.1
  • Remark 6.2
  • Remark 7.1