An accelerated gradient method with adaptive restart for convex multiobjective optimization problems

TL;DR

The paper develops an accelerated gradient framework for convex multiobjective optimization by deriving a rigorous continuous-time limit of multiobjective APG and introducing an Accelerated Multiobjective Gradient (AMG) flow with adaptive time scaling. An IMEX discretization yields an accelerated method with rates $\min\{L/k^2,(1-\sqrt{\mu/L})^k\}$, accompanied by a quadratic programming subproblem on the convex hull of gradients. A Lyapunov-based analysis proves exponential decay of a merit function, and practical gains are achieved via a residual-based adaptive restart to mitigate oscillations. The approach is validated numerically against state-of-the-art methods, showing competitive convergence in convex cases and linear-like behavior under strong convexity, with restart strategies significantly enhancing robustness and speed.

Abstract

In this work, based on the continuous time approach, we propose an accelerated gradient method with adaptive residual restart for convex multiobjective optimization problems. For the first, we derive rigorously the continuous limit of the multiobjective accelerated proximal gradient method by Tanabe et al. [Comput. Optim. Appl., 2023]. It is a second-order ordinary differential equation (ODE) that involves a special projection operator and can be viewed as an extension of the ODE by Su et al. [J. Mach. Learn. Res., 2016] for Nesterov acceleration. Then, we introduce a novel accelerated multiobjective gradient (AMG) flow with tailored time scaling that adapts automatically to the convex case and the strongly convex case, and the exponential decay rate of a merit function along with the solution trajectory of AMG flow is established via the Lyapunov analysis. After that, we consider an implicit-explicit time discretization and obtain an accelerated multiobjective gradient method with a convex quadratic programming subproblem. The fast sublinear rate and linear rate are proved respectively for convex and strongly convex problems. In addition, we present an efficient residual based adaptive restart technique to overcome the oscillation issue and improve the convergence significantly. Numerical results are provided to validate the practical performance of the proposed method.

Figures (8)

• Figure 1: The approximate Pareto front at the iteration $k = 25$, with $N=100$ initial sample points in $[-2,2]^n$ for the first example \ref{['eq:ex1']}.
• Figure 2: The KKT residual $\| {\bf proj}_{C(x_k)}(0)\|$ and the iterate gap $\left\lVert {x_{k+1}-x_k} \right\rVert$$v.s.$ the iteration step $k$, with one initial sample point $x_0\in[-2,2]^n$ for the first example \ref{['eq:ex1']}.
• Figure 3: The KKT residual $\| {\bf proj}_{C(x_k)}(0)\|$ and the iterate gap $\left\lVert {x_{k+1}-x_k} \right\rVert$$v.s.$ the iteration time $s$, with one initial sample point $x_0\in[-2,2]^n$ for the first example \ref{['eq:ex1']}.
• Figure 4: The KKT residual $\| {\bf proj}_{C(x_k)}(0)\|$ and the iterate gap $\left\lVert {x_{k+1}-x_k} \right\rVert$$v.s.$ the iteration step $k$, with one initial sample point $x_0\in[-2,2]^n$ for the second example \ref{['eq:ex2']}.
• Figure 5: The KKT residual $\| {\bf proj}_{C(x_k)}(0)\|$ and the iterate gap $\left\lVert {x_{k+1}-x_k} \right\rVert$$v.s.$ the iteration time $s$, with one initial sample point $x_0\in[-2,2]^n$ for the second example \ref{['eq:ex2']}.

Theorems & Definitions (27)

• Lemma 2.1
• proof
• Theorem 2.1: Tanabe2024
• Lemma 2.2
• proof
• Theorem 3.1
• Lemma 3.1
• proof
• Lemma 3.2
• proof