Convergence analysis of accelerated algorithms via a mixed-order dynamical system for separable nonsmooth convex optimization

Abstract

For a linear equality constrained convex optimization problem involving two objective functions with a ``nonsmooth" + ``nonsmooth" composite structure, we study two algorithms derived from a mixed-order dynamical system which incorporates time scales and a Tikhonov regularization term. We observe that different types of multipliers lead to distinct algorithms. For the implicit multiplier and semi-implicit multiplier, we develop a new primal-dual joint algorithm and a new splitting algorithm, respectively. Our proposed joint algorithm can reduce to an algorithm for solving the corresponding non-separable linearly constrained convex optimization problem. Then, we establish the nonergodic convergence properties of all our proposed algorithms. Moreover, we derive that the sequences generated by these algorithms strongly converge to the minimal norm solution. Finally, numerical experiments are conducted to validate the practical performance of the proposed algorithms.

Figures (5)

• Figure 1: The convergence behaviors of algorithms for problem (\ref{['szsy1']}) when $f$ is a convex function. The problem size is $(m, n) = (300, 3000)$.
• Figure 2: The convergence behaviors of algorithms for problem (\ref{['szsy1']}) when $f$ is a strongly convex function. The problem size is $(m, n) = (300, 3000)$.
• Figure 3: The convergence behaviors of algorithms for problem (\ref{['szsy1']}). Left: Case I (general convex); Right: Case II (partially strongly convex). The problem size is $(m, n) = (400, 5000)$.
• Figure 4: Numerical results of PDSA under the different choices of parameters with $\epsilon_k=0$.
• Figure 5: Numerical results of PDSA under the different choices of parameters with $\epsilon_k\neq 0$.

Theorems & Definitions (32)

• Proposition 1
• proof
• Proposition 2
• proof
• Remark 1
• Lemma 1
• Lemma 2
• proof
• Theorem 1
• proof