Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A continuous perspective on the inertial corrected primal-dual proximal splitting

Hao Luo

TL;DR

The paper develops a continuous-time perspective for the inertial corrected primal-dual proximal splitting (IC-PDPS) method applied to saddle-point problems. It introduces the notions of intrinsic and rescaled step sizes to handle the method's multiple evolving parameters and derives two second-order ODE models that are equivalent under a proper time transformation. A semi-implicit Euler reformulation yields a low- and a high-resolution continuous IC-PDPS dynamics with velocity-correction terms, and a tailored Lyapunov function proves nonincreasing energy and exponential decay of the Lagrangian gap. The results illuminate the continuous mechanisms behind IC-PDPS, connect discrete iterations to ODEs, and lay groundwork for future extensions such as non-Euclidean metrics, high-resolution/ Hessian-driven damping, and long-time error analysis.

Abstract

We give a continuous perspective on the Inertial Corrected Primal-Dual Proximal Splitting (IC-PDPS) proposed by Valkonen ({\it SIAM J. Optim.}, 30(2): 1391--1420, 2020) for solving saddle-point problems. The algorithm possesses nonergodic convergence rate and admits a tight preconditioned proximal point formulation which involves both inertia and additional correction. Based on new understandings on the relation between the discrete step size and rescaling effect, we rebuild IC-PDPS as a semi-implicit Euler scheme with respect to its iterative sequences and integrated parameters. This leads to two novel second-order ordinary differential equation (ODE) models that are equivalent under proper time transformation, and also provides an alternative interpretation from the continuous point of view. Besides, we present the convergence analysis of the Lagrangian gap along the continuous trajectory by using proper Lyapunov functions.

A continuous perspective on the inertial corrected primal-dual proximal splitting

TL;DR

The paper develops a continuous-time perspective for the inertial corrected primal-dual proximal splitting (IC-PDPS) method applied to saddle-point problems. It introduces the notions of intrinsic and rescaled step sizes to handle the method's multiple evolving parameters and derives two second-order ODE models that are equivalent under a proper time transformation. A semi-implicit Euler reformulation yields a low- and a high-resolution continuous IC-PDPS dynamics with velocity-correction terms, and a tailored Lyapunov function proves nonincreasing energy and exponential decay of the Lagrangian gap. The results illuminate the continuous mechanisms behind IC-PDPS, connect discrete iterations to ODEs, and lay groundwork for future extensions such as non-Euclidean metrics, high-resolution/ Hessian-driven damping, and long-time error analysis.

Abstract

We give a continuous perspective on the Inertial Corrected Primal-Dual Proximal Splitting (IC-PDPS) proposed by Valkonen ({\it SIAM J. Optim.}, 30(2): 1391--1420, 2020) for solving saddle-point problems. The algorithm possesses nonergodic convergence rate and admits a tight preconditioned proximal point formulation which involves both inertia and additional correction. Based on new understandings on the relation between the discrete step size and rescaling effect, we rebuild IC-PDPS as a semi-implicit Euler scheme with respect to its iterative sequences and integrated parameters. This leads to two novel second-order ordinary differential equation (ODE) models that are equivalent under proper time transformation, and also provides an alternative interpretation from the continuous point of view. Besides, we present the convergence analysis of the Lagrangian gap along the continuous trajectory by using proper Lyapunov functions.
Paper Structure (29 sections, 5 theorems, 142 equations, 3 figures, 1 algorithm)

This paper contains 29 sections, 5 theorems, 142 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature review
  3. Motivation
  4. Notation
  5. Main contribution
  6. Intrinsic step size v.s. rescaled step size
  7. The continuous model IC-PDPS
  8. Organization
  9. A General Corrected Inertial Framework
  10. The corrected inertial proximal point method
  11. The inertial corrected primal-dual proximal splitting
  12. Continuous-time NAG Revisited
  13. Intrinsic NAG ODE
  14. Rescaled NAG ODE
  15. The Continuous Model of IC-PDPS
  16. ...and 14 more sections

Key Result

Theorem 2.1

Let $\{u^i:i\in\mathbb N\}\subset U$ be generated from eq:CP-PPA-mod-IC and the auxiliary sequence $\{z^i:i\in\mathbb N\}\subset\mathcal{U}$ be defined by eq:zi, with the initial state $u^0=\bar{u}^0\in U$, the preconditioners $\{M_{i+1}:i\in\mathbb N\}\subset \mathcal{L}( U ; U )$ and the invertib then where $\Upsilon_{i+1}: = \Lambda_{i+1}^* Z_{i+1}M_{i+1} \Lambda_{i+1}$ is self-adjoint and po

Figures (3)

  • Figure 1: Numerical illustration of \ref{['eq:NAG']}, intrinsic NAG ODE \ref{['eq:NAG-int-ode']} and rescaled NAG ODE \ref{['eq:NAG-res-ode']} with the same initial values. The objective is simply quadratic $G(x)=x^2/2$ in one dimension. In the top line, we see smaller $\tau$ leads to better approximations of the continuous trajectories. In the middle line, we also report the convergence behavior of the objective. In the bottom line, $X(t)$ and $\widetilde{X}(s)$ are different in terms of their own time coordinates but under the rescale relations $t(s)=2\ln(1+0.5s/\theta(0))$ and $s(t)=2\theta(0)(e^{t/2}-1)$, they coincide with each other. This also agrees with what we have observed from the phase space (top line).
  • Figure 2: Numerical illustration of the trajectories $X^i=(x^i,y^i,\zeta^i,\eta^i)$ for \ref{['algo:IC-PDPS']} and $\widetilde{X}(s)=(\widetilde{x}(s),\widetilde{y}(s),\widetilde{\zeta}(s),\widetilde{\eta}(s))$ for the intrinsic IC-PDPS ODE \ref{['eq:int-ode-ic-pdps']}. We simply take $K=1$ and quadratic objectives: $G(x)=x^2/2,\,F^*(y) = y^2/2$ in one dimension. From the left and middle, we see smaller $\alpha$ leads to better approximations of the continuous trajectories. In the right, we also report the convergence behavior of the objective in terms of the time $s_i=\alpha i\in[0,20]$.
  • Figure 3: Numerical illustration of the trajectories $X^i=(x^i,y^i,\zeta^i,\eta^i)$ for \ref{['algo:IC-PDPS']} and $X(t)=(x(t),y(t),\zeta(t),\eta(t))$ for the rescaled IC-PDPS ODE \ref{['eq:ode-ic-pdps']}. The objectives are similar with that in \ref{['fig:icpdps-intode']}. From the left and middle, we observe that $X(t)$ and $\widetilde{X}(s)$ are identical in the phase space. In the right, we show the convergence behavior of the objective with respect to the rescaled time $t_i$, which is related to the intrinsic time $s_i$ by the transformation \ref{['eq:time-t-ic-pdps']}.

Theorems & Definitions (15)

  • Theorem 2.1: valkonen_inertial_2020
  • Remark 2.1
  • Remark 3.1
  • Remark 3.2
  • Remark 3.3
  • Lemma 4.1
  • proof
  • Remark 4.1
  • Theorem 4.1
  • proof
  • ...and 5 more