Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated Mirror Descent Method through Variable and Operator Splitting

Long Chen, Hao Luo, Jingrong Wei, Zeyi Xu, Yuan Yao

TL;DR

This work develops Accelerated Mirror Descent (Acc-MD) by discretizing a Variable–Operator Splitting (VOS) flow in non-Euclidean, Bregman geometries. A new Generalized Cauchy–Schwarz (GCS) condition, with constant $C_{f,\phi}$, enables the first accelerated linear convergence for mirror-descent-based methods under relative smoothness/convexity, with a geometric Lyapunov analysis underpinning the results. The paper extends Acc-MD to convex and composite settings, using perturbation and homotopy to recover the optimal $\mathcal{O}(C_{f,\phi}/k^2)$ rate, and demonstrates substantial empirical gains on quartic, entropic, and LASSO problems. It highlights the practical impact of non-Euclidean acceleration for constrained and non-smooth problems, and outlines directions for adaptive $C_{f,\phi}$ and stochastic extensions. Overall, the framework broadens the applicability of accelerated first-order methods to a wide class of Bregman geometries while providing rigorous convergence guarantees.

Abstract

Mirror descent uses the mirror function to encode geometry and constraints, improving convergence while preserving feasibility. Accelerated Mirror Descent Methods (Acc-MD) are derived from a discretization of an accelerated mirror ODE system using a variable--operator splitting framework. A geometric assumption, termed the Generalized Cauchy-Schwarz (GCS) condition, is introduced to quantify the compatibility between the objective and the mirror geometry, under which the first accelerated linear convergence for Acc-MD on a broad class of problems is established. Numerical experiments on smooth and composite optimization tasks demonstrate that Acc-MD consistently outperforms existing accelerated variants, both theoretically and empirically.

Accelerated Mirror Descent Method through Variable and Operator Splitting

TL;DR

This work develops Accelerated Mirror Descent (Acc-MD) by discretizing a Variable–Operator Splitting (VOS) flow in non-Euclidean, Bregman geometries. A new Generalized Cauchy–Schwarz (GCS) condition, with constant , enables the first accelerated linear convergence for mirror-descent-based methods under relative smoothness/convexity, with a geometric Lyapunov analysis underpinning the results. The paper extends Acc-MD to convex and composite settings, using perturbation and homotopy to recover the optimal rate, and demonstrates substantial empirical gains on quartic, entropic, and LASSO problems. It highlights the practical impact of non-Euclidean acceleration for constrained and non-smooth problems, and outlines directions for adaptive and stochastic extensions. Overall, the framework broadens the applicability of accelerated first-order methods to a wide class of Bregman geometries while providing rigorous convergence guarantees.

Abstract

Mirror descent uses the mirror function to encode geometry and constraints, improving convergence while preserving feasibility. Accelerated Mirror Descent Methods (Acc-MD) are derived from a discretization of an accelerated mirror ODE system using a variable--operator splitting framework. A geometric assumption, termed the Generalized Cauchy-Schwarz (GCS) condition, is introduced to quantify the compatibility between the objective and the mirror geometry, under which the first accelerated linear convergence for Acc-MD on a broad class of problems is established. Numerical experiments on smooth and composite optimization tasks demonstrate that Acc-MD consistently outperforms existing accelerated variants, both theoretically and empirically.
Paper Structure (21 sections, 15 theorems, 108 equations, 4 figures, 1 table, 5 algorithms)

This paper contains 21 sections, 15 theorems, 108 equations, 4 figures, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Mirror Descent Methods
  4. Accelerated Mirror Descent Methods
  5. Flow and stability
  6. Strong Lyapunov condition for linear cases
  7. Strong Lyapunov condition of nonlinear cases
  8. Accelerated Mirror Descent Methods
  9. Convergence analysis
  10. Convergence analysis of linear cases
  11. Convergence analysis of nonlinear cases
  12. Discussion on the generalized Cauchy--Schwarz condition
  13. Comparison between \ref{['assump: A2']} and TSE
  14. Extension to convex optimization
  15. Convergence Analysis
  16. ...and 6 more sections

Key Result

Lemma 3.1

Let $\{x_k\}$ be the sequence generated by the mirror descent method eq:md. Then

Figures (4)

  • Figure 1: Comparison of assumptions. For the entropy mirror, Acc-MD achieves acceleration under GCS, whereas ABPG fails due to the lack of a suitable TSE.
  • Figure 1: Log relative error vs. running time for the quartic problem.
  • Figure 2: Log relative error vs. running time curve on the entropic MD problem.
  • Figure 3: Relative error vs. running time curve for the LASSO problem.

Theorems & Definitions (31)

  • Lemma 3.1
  • Proof 1
  • Theorem 3.2
  • Proof 2
  • Lemma 4.1
  • Proof 3
  • Lemma 4.2
  • Proof 4
  • Lemma 4.3
  • Proof 5
  • ...and 21 more