Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MGProx: A nonsmooth multigrid proximal gradient method with adaptive restriction for strongly convex optimization

Andersen Ang, Hans De Sterck, Stephen Vavasis

TL;DR

In the numerical tests on the Elastic Obstacle Problem, which is an example of nonsmooth convex optimization problem where multigrid method can be applied, it is shown that MGProx has a faster convergence speed than competing methods.

Abstract

We study the combination of proximal gradient descent with multigrid for solving a class of possibly nonsmooth strongly convex optimization problems. We propose a multigrid proximal gradient method called MGProx, which accelerates the proximal gradient method by multigrid, based on using hierarchical information of the optimization problem. MGProx applies a newly introduced adaptive restriction operator to simplify the Minkowski sum of subdifferentials of the nondifferentiable objective function across different levels. We provide a theoretical characterization of MGProx. First we show that the MGProx update operator exhibits a fixed-point property. Next, we show that the coarse correction is a descent direction for the fine variable of the original fine level problem in the general nonsmooth case. Lastly, under some assumptions we provide the convergence rate for the algorithm. In the numerical tests on the Elastic Obstacle Problem, which is an example of nonsmooth convex optimization problem where multigrid method can be applied, we show that MGProx has a faster convergence speed than competing methods.

MGProx: A nonsmooth multigrid proximal gradient method with adaptive restriction for strongly convex optimization

TL;DR

In the numerical tests on the Elastic Obstacle Problem, which is an example of nonsmooth convex optimization problem where multigrid method can be applied, it is shown that MGProx has a faster convergence speed than competing methods.

Abstract

We study the combination of proximal gradient descent with multigrid for solving a class of possibly nonsmooth strongly convex optimization problems. We propose a multigrid proximal gradient method called MGProx, which accelerates the proximal gradient method by multigrid, based on using hierarchical information of the optimization problem. MGProx applies a newly introduced adaptive restriction operator to simplify the Minkowski sum of subdifferentials of the nondifferentiable objective function across different levels. We provide a theoretical characterization of MGProx. First we show that the MGProx update operator exhibits a fixed-point property. Next, we show that the coarse correction is a descent direction for the fine variable of the original fine level problem in the general nonsmooth case. Lastly, under some assumptions we provide the convergence rate for the algorithm. In the numerical tests on the Elastic Obstacle Problem, which is an example of nonsmooth convex optimization problem where multigrid method can be applied, we show that MGProx has a faster convergence speed than competing methods.
Paper Structure (67 sections, 18 theorems, 96 equations, 1 figure, 1 table, 5 algorithms)

This paper contains 67 sections, 18 theorems, 96 equations, 1 figure, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Modern models are nonsmooth
  3. Classical problems in scientific computing are smooth
  4. This work: bridging smoothness and nonsmoothness
  5. Classical multigrid and notation
  6. Notation
  7. MGOPT
  8. Proximal gradient method
  9. Contributions
  10. Literature review
  11. Early works
  12. Two families of multigrid
  13. How our approach differs
  14. Multigrid outside PDEs
  15. Multigrid in image processing
  16. ...and 52 more sections

Key Result

Lemma 2.2

\newlabellem:strcvx_linearmap0 Given a function $F : \mathbb{R}^n \rightarrow \overline{\mathbb{R}}$ that is $\mu$-strongly convex and a rank-$n$ matrix $R \in \mathbb{R}^{m \times n}$, the function $F \circ R$ is $\mu \sigma^2_n$-strongly convex, where $\sigma_n$ is the $n$th singular value of $R

Figures (1)

  • Figure 1: The convergence pattern of the algorithms for the case $N^2=225$. Left: x-axis in log scale. Right: x-axis in linear scale. FISTAr has the fastest convergence in the first 10 seconds, then MGProx has the fastest convergence over all.

Theorems & Definitions (45)

  • Remark 1.1: MGOPT has no theoretical convergence guarantee
  • Definition 2.1: Restriction
  • Lemma 2.2: Composition with full-rank matrix preserves convexity
  • Proof 1
  • Definition 2.3: Adaptive restriction operator for separable $g$
  • Theorem 2.4: Fixed-point
  • Proof 2
  • Theorem 2.5: Angle condition of coarse correction
  • Proof 3
  • Remark 2.6
  • ...and 35 more