Multilevel Geometric Optimization for Regularised Constrained Linear Inverse Problems

TL;DR

The paper tackles accelerating optimization for box-constrained convex problems by developing a geometric multilevel framework. It builds a hierarchy of discretizations and uses a coarse-grid model defined on a Riemannian manifold structure of the interior of the box to compute descent directions efficiently, while preserving feasibility. Key contributions include a geometry-aware coarse objective, tangent-space transfers between levels, and Armijo line search on manifolds, demonstrated on a KL-divergence plus smoothed TV regularized inverse problem where the coarse grid accelerates convergence relative to state-of-the-art first-order methods. The approach leverages information geometry to extend multigrid ideas to constrained optimization, enabling scalable solutions for large-scale inverse problems with box constraints.

Abstract

We present a geometric multilevel optimization approach that smoothly incorporates box constraints. Given a box constrained optimization problem, we consider a hierarchy of models with varying discretization levels. Finer models are accurate but expensive to compute, while coarser models are less accurate but cheaper to compute. When working at the fine level, multilevel optimisation computes the search direction based on a coarser model which speeds up updates at the fine level. Moreover, exploiting geometry induced by the hierarchy the feasibility of the updates is preserved. In particular, our approach extends classical components of multigrid methods like restriction and prolongation to the Riemannian structure of our constraints.

Figures (6)

• Figure 1.1: Left. Incidence geometry of projection rays and cells indexed by $I_{n}$, that cover a subset in $\mathbb{R}^{d}$ (here: $d=2$ for illustration). The corresponding line integrals define the matrix $A$ that linearly maps cell values $y_{i},\, i\in I_{n}$ to projections $b_{j},\, j\in \{1,\dotsc,p\}$. This gives rise to an underdetermined linear system $A y = b$. The task is to recover $y$ from $b$ by solving \ref{['eq:intro-approach']} using the constraints and the regularizer to obtain a well-posed problem. Right. Reducing the spatial resolution yields a coarse level representation of the projection matrix, that enables to evaluate a surrogate of the objective function on the coarse level which does not involve the data vector $b$ recorded on the fine level. This coarse level representation allows to compute descent directions on the fine level efficiently using a smaller subset of variables at the coarse level.
• Figure 5.1: The phantoms ($1024\times1024$) used for the numerical evaluation exhibit both fine and large scale structures and various shapes.
• Figure 5.2: ABPG vs. Riemannian gradient (RG) descent in terms of relative objective values is compared on 2% and 20% undersampled projection data. The left column corresponds to the first three phantoms: gear, bone, vessel in Figure \ref{['fig:phantoms']}, the right columns to the last three: batenburg, roux, skulls. The Riemannian gradient descent achives lower objective values due to proper step-size selection (line search). For the more homogeneous phantoms corresponding to the right column, the proposed method significantly outperforms ABPG Hanzely:2021vc.
• Figure 5.3: 20% undersampling: comparison of relative objective function values for single-level resp. two-level (2L) Riemannian gradient descent (RG, 2L RG) and ABPG Hanzely:2021vc. The left column corresponds to the first three phantoms: gear, bone, vessel in Figure \ref{['fig:phantoms']}, the right columns to the last three: batenburg, roux, skulls. Black dots indicate when descent directions were computed on coarser grids. On the coarse grid we have 40% undersampling and the coarse problem has a unique solution Kuske2019. As a consequence we quickly approach the boundary of the box. This results in an inefficient update using the Armijo line search and more similar curves for single-level resp. two-level (2L) Riemannian gradient descent.
• Figure 5.4: 2% undersampling: comparison of relative objective function values for single-level resp. two-level (2L) Riemannian gradient descent (RG, 2L RG) and ABPG Hanzely:2021vc. The left column corresponds to the first three phantoms: gear, bone, vessel in Figure \ref{['fig:phantoms']}, the right columns to the last three: batenburg, roux, skulls. Black dots indicate when descent directions were computed on coarser grids. The two-level schemes - where we now have 4% undersampling - aggressively decreases the objective, in particular for more homogeneous phantoms in the right column.

Theorems & Definitions (26)

• Definition 2.1: Bregman divergence Bregman:1967tn, Censor:1981vy, Borwein:2010aa
• Lemma 2.1: Chen_Teboulle_1993
• Lemma 2.2
• proof
• Remark 3.1
• Lemma 3.1
• proof
• Remark 3.2
• Proposition 4.1
• proof