Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FISTA-Condat-Vu: Automatic Differentiation for Hyperparameter Learning in Variational Models

Patricio Guerrero, Simon Bellens, Wim Dewulf

TL;DR

This work addresses learning the regularization weight $\lambda$ in non-smooth variational imaging models for ill-posed inverse problems, notably few-view industrial CT. It introduces a memory-efficient bilevel learning framework by merging FISTA with Condat-Vu and a reduced-memory automatic differentiation strategy, including an assisted CV (aCV) variant. By deriving $\nabla_\lambda L$ with respect to $\lambda$ and employing an NGD outer loop with Armijo backtracking, the approach achieves efficient hyperparameter learning with significantly reduced memory footprints, demonstrated on large-scale 3D CT data. The proposed method enables automatic, scalable hyperparameter tuning for high-dimensional, non-smooth regularizers, with practical impact on industrial imaging tasks and accessible code.

Abstract

Motivated by industrial computed tomography, we propose a memory efficient strategy to estimate the regularization hyperparameter of a non-smooth variational model. The approach is based on a combination of FISTA and Condat-Vu algorithms exploiting the convergence rate of the former and the low per-iteration complexity of the latter. The estimation is cast as a bilevel learning problem where a first-order method is obtained via reduced-memory automatic differentiation to compute the derivatives. The method is validated with experimental industrial tomographic data with the numerical implementation available.

FISTA-Condat-Vu: Automatic Differentiation for Hyperparameter Learning in Variational Models

TL;DR

This work addresses learning the regularization weight in non-smooth variational imaging models for ill-posed inverse problems, notably few-view industrial CT. It introduces a memory-efficient bilevel learning framework by merging FISTA with Condat-Vu and a reduced-memory automatic differentiation strategy, including an assisted CV (aCV) variant. By deriving with respect to and employing an NGD outer loop with Armijo backtracking, the approach achieves efficient hyperparameter learning with significantly reduced memory footprints, demonstrated on large-scale 3D CT data. The proposed method enables automatic, scalable hyperparameter tuning for high-dimensional, non-smooth regularizers, with practical impact on industrial imaging tasks and accessible code.

Abstract

Motivated by industrial computed tomography, we propose a memory efficient strategy to estimate the regularization hyperparameter of a non-smooth variational model. The approach is based on a combination of FISTA and Condat-Vu algorithms exploiting the convergence rate of the former and the low per-iteration complexity of the latter. The estimation is cast as a bilevel learning problem where a first-order method is obtained via reduced-memory automatic differentiation to compute the derivatives. The method is validated with experimental industrial tomographic data with the numerical implementation available.

Paper Structure

This paper contains 17 sections, 2 theorems, 16 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Notations
  3. Problem setting
  4. Motivating application
  5. Related work
  6. Contributions
  7. Image reconstruction
  8. FISTA and Condat-Vu
  9. Preliminary application: the proximity operator of $\mathrm{tv} + \imath_{\mathbb{R}_+}$
  10. Equivalence between GP and CV for denoising
  11. Main result: hyperparameter learning
  12. Derivative of the loss function
  13. Sensitivity of $\lambda$ in GP and CV
  14. assisted Condat-Vu and the FISTA-aCV algorithm
  15. The resulting first-order algorithm for $\lambda$
  16. ...and 2 more sections

Key Result

Proposition 1

GP is a dual instance of CV applied on the TV denoising Problem (eq.denoising)

Figures (3)

  • Figure 1: Computational graph of a GP iteration: the map $w_k \mapsto w_{k+1}$.
  • Figure 2: Computational graph of a CV iteration: the map $(u_k,w_k) \mapsto (u_{k+1},w_{k+1})$. Note that the constants $\{\gamma, \sigma\}$ do not depend on $\lambda$.
  • Figure 3: Industrial CT experiment. From left to right: ground-truth (FDK with 3142 projections), FDK with 60 projections, FISTA-aCV with 60 projections. Top: $1400^2$ pixels. Bottom: $400^2$ pixels.

Theorems & Definitions (2)

  • Proposition 1
  • Proposition 2