Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep learning methods for inverse problems using connections between proximal operators and Hamilton-Jacobi equations

Oluwatosin Akande, Gabriel P. Langlois, Akwum Onwunta

TL;DR

The paper addresses the challenge of solving ill-posed inverse problems by learning priors through proximal operators and their deep connections to Hamilton–Jacobi equations. It formalizes how the viscosity solution framework links $S(oldsymbol{x},t)$, the proximal operator, and the initial data $J$, enabling learning of the backward viscosity solution prior $J_{ ext{BVS}}$ from data. The authors develop a learning pipeline using convex neural networks and max-plus approximations to reconstruct $J_{ ext{BVS}}$ from incomplete information, and demonstrate numerical results across high-dimensional settings for convex, nonconvex, concave, and negative-norm priors. The work situates proximal methods within a broader PDE-based learning paradigm, offering principled error bounds and scalable approaches for learning priors in inverse problems.

Abstract

Inverse problems are important mathematical problems that seek to recover model parameters from noisy data. Since inverse problems are often ill-posed, they require regularization or incorporation of prior information about the underlying model or unknown variables. Proximal operators, ubiquitous in nonsmooth optimization, are central to this because they provide a flexible and convenient way to encode priors and build efficient iterative algorithms. They have also recently become key to modern machine learning methods, e.g., for plug-and-play methods for learned denoisers and deep neural architectures for learning priors of proximal operators. The latter was developed partly due to recent work characterizing proximal operators of nonconvex priors as subdifferential of convex potentials. In this work, we propose to leverage connections between proximal operators and Hamilton-Jacobi partial differential equations (HJ PDEs) to develop novel deep learning architectures for learning the prior. In contrast to other existing methods, we learn the prior directly without recourse to inverting the prior after training. We present several numerical results that demonstrate the efficiency of the proposed method in high dimensions.

Deep learning methods for inverse problems using connections between proximal operators and Hamilton-Jacobi equations

TL;DR

The paper addresses the challenge of solving ill-posed inverse problems by learning priors through proximal operators and their deep connections to Hamilton–Jacobi equations. It formalizes how the viscosity solution framework links , the proximal operator, and the initial data , enabling learning of the backward viscosity solution prior from data. The authors develop a learning pipeline using convex neural networks and max-plus approximations to reconstruct from incomplete information, and demonstrate numerical results across high-dimensional settings for convex, nonconvex, concave, and negative-norm priors. The work situates proximal methods within a broader PDE-based learning paradigm, offering principled error bounds and scalable approaches for learning priors in inverse problems.

Abstract

Inverse problems are important mathematical problems that seek to recover model parameters from noisy data. Since inverse problems are often ill-posed, they require regularization or incorporation of prior information about the underlying model or unknown variables. Proximal operators, ubiquitous in nonsmooth optimization, are central to this because they provide a flexible and convenient way to encode priors and build efficient iterative algorithms. They have also recently become key to modern machine learning methods, e.g., for plug-and-play methods for learned denoisers and deep neural architectures for learning priors of proximal operators. The latter was developed partly due to recent work characterizing proximal operators of nonconvex priors as subdifferential of convex potentials. In this work, we propose to leverage connections between proximal operators and Hamilton-Jacobi partial differential equations (HJ PDEs) to develop novel deep learning architectures for learning the prior. In contrast to other existing methods, we learn the prior directly without recourse to inverting the prior after training. We present several numerical results that demonstrate the efficiency of the proposed method in high dimensions.
Paper Structure (24 sections, 7 theorems, 66 equations, 5 figures, 4 tables)

This paper contains 24 sections, 7 theorems, 66 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Proximal operators
  4. Hamilton--Jacobi Equations
  5. Characteristic equations
  6. Viscosity solutions and the Lax--Oleinik formula
  7. Convex neural networks
  8. Connections between learning priors and the inverse problem for Hamilton--Jacobi Equations
  9. Reachability and inverse problems for Hamilton--Jacobi equations
  10. Learning priors and the inverse problem for Hamilton--Jacobi Equations with incomplete information
  11. Piecewise approximations
  12. Piecewise affine approximation
  13. Piecewise quadratic approximation
  14. Max-plus algebra theory for Hamilton--Jacobi PDEs and approximation results
  15. Applications to the inverse problem for Hamilton--Jacobi Equations
  16. ...and 9 more sections

Key Result

Theorem 1

A proper function $f_{t}\colon \mathbb{R}^n\to \mathbb{R}^n$ is a proximal operator of $tJ$ if and only if $\boldsymbol{x} \mapsto \psi(\boldsymbol{x},t)$ is proper, lower semicontinuous, and convex and $f_{t}(\boldsymbol{x}) \in \partial_{\boldsymbol{x}}\psi(\boldsymbol{x},t)$. Moreover, $f_{t}$ is

Figures (5)

  • Figure 1: The cross sections of the convex function $\psi(x)$ for dimension $8$ (top). The bottom row compares the cross sections of the prior function from "invert LPN" (left) and our trained second LPN method (right).
  • Figure 2: The cross sections of the convex function $\psi(x)$ for dimension $8$ (top). The bottom row compares the cross sections of the prior function from "invert LPN" (left) and our trained second LPN method (right).
  • Figure 3: The cross sections of the convex function $\psi(x)$ for dimension $32$ (top). The bottom row compares the cross sections of the prior function from "invert LPN" (left) and our trained second LPN method (right)
  • Figure 4: The cross sections of the prior function for $8$ dimension from "invert LPN" (left) and our trained second LPN method (right)
  • Figure 5: The cross sections of the convex function $\psi(x)$ for dimension $4$ (left) and $8$ (right).

Theorems & Definitions (19)

  • Theorem 1
  • Proof 1
  • Definition 2
  • Remark 2.1
  • Example 2.1: The negative absolute value prior
  • Example 2.2
  • Example 2.3: The negative absolute value prior, continued.
  • Theorem 1
  • Proof 2
  • Theorem 2
  • ...and 9 more