Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A General Tikhonov Regularized Second-Order Dynamical System for Convex-Concave Bilinear Saddle Point Problems

Bohan Zhang, Xiaojun Zhang

TL;DR

This work addresses convex-concave bilinear saddle point problems via a general Tikhonov-regularized, second-order primal-dual dynamical system that includes damping, time scaling, and extrapolation. A Lyapunov-based framework yields two main asymptotic regimes for the Tikhonov parameter ε(t): rapid decay gives a primal-dual gap rate of $O\left(\frac{1}{t^2\beta(t)}\right)$, while slow decay gives $o\left(\frac{1}{\beta(t)}\right)$, along with strong convergence of the trajectory to the minimum-norm saddle point and integral estimates. The paper also provides several particular cases with explicit parameter choices, demonstrating fast convergence and robustness, and validates the theory through comprehensive numerical experiments that show improvements over existing accelerated primal-dual dynamics. Overall, the framework advances understanding of how Tikhonov regularization interacts with second-order dynamics to achieve both fast convergence and strong convergence in saddle-point problems.

Abstract

In this paper, we propose a general Tikhonov regularized second-order dynamical system with viscous damping, time scaling and extrapolation coefficients for the convex-concave bilinear saddle point problem. By the Lyapunov function approach, we show that the convergence properties of the proposed dynamical system depend on the choice of the Tikhonov regularization parameter. Specifically, when the Tikhonov regularization parameter tends to zero rapidly, the convergence rate of the primal-dual gap along the generated trajectory is O(1 over t squared times beta(t)); when the Tikhonov regularization parameter tends to zero slowly, the convergence rate of the primal-dual gap is o(1 over beta(t)). We also prove the strong convergence property of the trajectory generated by the Tikhonov regularized dynamical system to the minimum-norm solution of the convex-concave bilinear saddle point problem, and derive several integral estimates. In addition, the effectiveness of the proposed dynamical system is verified through a series of numerical experiments.

A General Tikhonov Regularized Second-Order Dynamical System for Convex-Concave Bilinear Saddle Point Problems

TL;DR

This work addresses convex-concave bilinear saddle point problems via a general Tikhonov-regularized, second-order primal-dual dynamical system that includes damping, time scaling, and extrapolation. A Lyapunov-based framework yields two main asymptotic regimes for the Tikhonov parameter ε(t): rapid decay gives a primal-dual gap rate of , while slow decay gives , along with strong convergence of the trajectory to the minimum-norm saddle point and integral estimates. The paper also provides several particular cases with explicit parameter choices, demonstrating fast convergence and robustness, and validates the theory through comprehensive numerical experiments that show improvements over existing accelerated primal-dual dynamics. Overall, the framework advances understanding of how Tikhonov regularization interacts with second-order dynamics to achieve both fast convergence and strong convergence in saddle-point problems.

Abstract

In this paper, we propose a general Tikhonov regularized second-order dynamical system with viscous damping, time scaling and extrapolation coefficients for the convex-concave bilinear saddle point problem. By the Lyapunov function approach, we show that the convergence properties of the proposed dynamical system depend on the choice of the Tikhonov regularization parameter. Specifically, when the Tikhonov regularization parameter tends to zero rapidly, the convergence rate of the primal-dual gap along the generated trajectory is O(1 over t squared times beta(t)); when the Tikhonov regularization parameter tends to zero slowly, the convergence rate of the primal-dual gap is o(1 over beta(t)). We also prove the strong convergence property of the trajectory generated by the Tikhonov regularized dynamical system to the minimum-norm solution of the convex-concave bilinear saddle point problem, and derive several integral estimates. In addition, the effectiveness of the proposed dynamical system is verified through a series of numerical experiments.
Paper Structure (12 sections, 10 theorems, 120 equations, 7 figures)

This paper contains 12 sections, 10 theorems, 120 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Preliminary Knowledge
  3. The Properties of Saddle Point Problems and Their Augmented Forms
  4. Preliminary Results and Cauchy Well-Posedness
  5. Convergence Analysis
  6. Asymptotic Analysis
  7. Case $\int_{t_0}^{+\infty} t \beta(t) \epsilon(t) \, \mathrm{d}t < +\infty$
  8. Case $\int_{t_0}^{+\infty} \frac{\beta(t)\epsilon(t)}{t} \, \mathrm{d}t < +\infty$
  9. Strong Convergence of the Trajectory
  10. Particular Cases
  11. Numercial Experiments
  12. Conclusion

Key Result

Lemma 2.1

ref38 Let $\theta>0$, $a \in \mathbb{R}^n$, and $x:[t_0,+\infty) \to \mathbb{R}^n$ is continuously differentiable. If there exists a constant $D_0$ such that then, $x(t)$ is bounded on the interval $[t_0,+\infty)$.

Figures (7)

  • Figure 1: Convergence of trajectories with different terms
  • Figure 2: Error analysis of the dynamical system \ref{['eq15']} under different parameters $r$ for the problem \ref{['e1']}
  • Figure 3: Comparison of error results between the dynamical system \ref{['eq15']}, APDD and Sun
  • Figure 4: Convergence of $\Phi$ with $m = 100$ and $n = 200$
  • Figure 5: Convergence of $\Phi$ with $m = 200$ and $n = 500$
  • ...and 2 more figures

Theorems & Definitions (26)

  • Definition 2.1
  • Lemma 2.1
  • Lemma 2.2
  • Definition 2.2
  • Theorem 2.1
  • proof
  • Lemma 3.1
  • proof
  • Theorem 3.1
  • proof
  • ...and 16 more