Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tikhonov regularization of second-order plus first-order primal-dual dynamical systems for separable convex optimization

Xiangkai Sun, Lijuan Zheng, Kok Lay Teo

TL;DR

This work develops a Tikhonov-regularized second-order plus first-order primal-dual dynamical system with time scaling to solve separable convex optimization problems with linear equality constraints. Using Lyapunov analysis, it establishes convergence rates for the primal-dual gap, objective error, feasibility measure, and gradient norms, with the Lagrangian gap decaying as $O\left(\frac{1}{\beta(t)}\right)$ and the others as $O\left(\frac{1}{\sqrt{\beta(t)}}\right)$; under $\beta(t)\epsilon(t)\to\infty$ and $\int\epsilon<\infty$, the primal trajectory converges strongly to the minimal-norm solution. The polynomial choices $\beta(t)=t^{r_1}$ and $\epsilon(t)=c/t^{r_2}$ yield refined rates depending on $r_2$. Numerical experiments show faster convergence than recent inertial primal-dual dynamics, validating the theoretical rates and the benefit of the Tikhonov regularization and time scaling. The results extend to smooth+nonsmooth settings via Moreau envelopes and contribute to designing fast continuous-time solvers for constrained convex optimization.

Abstract

This paper deals with a Tikhonov regularized second-order plus first-order primal-dual dynamical system with time scaling for separable convex optimization problems with linear equality constraints. This system consists of two second-order ordinary differential equations for the primal variables and one first-order ordinary differential equation for the dual variable.By utilizing the Lyapunov analysis approach, we obtain the convergence properties of the primal-dual gap, the objective function error, the feasibility measure and the gradient norm of the objective function along the trajectory. We also establish the strong convergence of the primal trajectory generated by the dynamical system towards the minimal norm solution of the separable convex optimization problem. Furthermore, we give numerical experiments to illustrate the theoretical results, showing that our dynamical system performs better than those in the literature in terms of convergence rates.

Tikhonov regularization of second-order plus first-order primal-dual dynamical systems for separable convex optimization

TL;DR

This work develops a Tikhonov-regularized second-order plus first-order primal-dual dynamical system with time scaling to solve separable convex optimization problems with linear equality constraints. Using Lyapunov analysis, it establishes convergence rates for the primal-dual gap, objective error, feasibility measure, and gradient norms, with the Lagrangian gap decaying as and the others as ; under and , the primal trajectory converges strongly to the minimal-norm solution. The polynomial choices and yield refined rates depending on . Numerical experiments show faster convergence than recent inertial primal-dual dynamics, validating the theoretical rates and the benefit of the Tikhonov regularization and time scaling. The results extend to smooth+nonsmooth settings via Moreau envelopes and contribute to designing fast continuous-time solvers for constrained convex optimization.

Abstract

This paper deals with a Tikhonov regularized second-order plus first-order primal-dual dynamical system with time scaling for separable convex optimization problems with linear equality constraints. This system consists of two second-order ordinary differential equations for the primal variables and one first-order ordinary differential equation for the dual variable.By utilizing the Lyapunov analysis approach, we obtain the convergence properties of the primal-dual gap, the objective function error, the feasibility measure and the gradient norm of the objective function along the trajectory. We also establish the strong convergence of the primal trajectory generated by the dynamical system towards the minimal norm solution of the separable convex optimization problem. Furthermore, we give numerical experiments to illustrate the theoretical results, showing that our dynamical system performs better than those in the literature in terms of convergence rates.
Paper Structure (6 sections, 10 theorems, 149 equations)

This paper contains 6 sections, 10 theorems, 149 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Existence and Uniqueness of the Solution Trajectory
  4. Asymptotic Properties of the Dynamical System (\ref{['1.3']})
  5. Strong Convergence Results
  6. Numerical Experiments

Key Result

lemma thmcounterlemma

ACr2018 Suppose that $\delta>0$, $\phi \in {L}^1([ \delta ,+\infty))$ is a nonnegative and continuous function, and $\psi :[ \delta ,+\infty)\rightarrow (0,+\infty )$ is a nondecreasing function such that $\lim_{t\to+\infty}\psi (t )=+\infty$. Then,

Theorems & Definitions (21)

  • lemma thmcounterlemma
  • theorem 1
  • proof
  • theorem 2
  • proof
  • remark thmcounterremark
  • theorem 3
  • proof
  • theorem 4
  • proof
  • ...and 11 more