Fast Reflected Forward-Backward algorithm: achieving fast convergence rates for convex optimization with linear cone constraints

TL;DR

The work develops a fast, momentum-augmented Reflected Forward-Backward scheme (Fast RFB) for monotone inclusions of the form $0\in M(z)+F(z)$, achieving weak convergence and a nonergodic last-iterate rate of $o\left( \dfrac{1}{k} \right)$ for both the discrete velocity and the tangent residual. Building on Fast RFB, the authors derive a fully splitting primal-dual algorithm applicable to saddle-point problems and convex optimizations with linear cone constraints, with the same nonergodic $o(1/k)$ rates for velocities, primal-dual gaps, and feasibility measures. The analysis combines a Lyapunov-type approach with careful parameter choices $(\alpha>2,$ $\frac{\alpha}{2}<c<\alpha-1,$ $0<\gamma<\frac{1}{2L})$ to establish convergence and rates, and the numerical experiments demonstrate improved practical performance over established methods such as EG, OGDA, FRB, RFB, PEAG, and ARG. These results contribute a robust, fully splitting, accelerated framework for broad classes of monotone inclusions and minimax problems, including those with linear cone constraints, with strong theoretical guarantees and empirical validation.

Abstract

In this paper, we derive a Fast Reflected Forward-Backward (Fast RFB) algorithm to solve the problem of finding a zero of the sum of a maximally monotone operator and a monotone and Lipschitz continuous operator in a real Hilbert space. Our approach extends the class of reflected forward-backward methods by introducing a Nesterov momentum term and a correction term, resulting in enhanced convergence performance. The iterative sequence of the proposed algorithm is proven to converge weakly, and the Fast RFB algorithm demonstrates impressive convergence rates, achieving $o\left( \frac{1}{k} \right)$ as $k \to +\infty$ for both the discrete velocity and the tangent residual at the \emph{last-iterate}. When applied to minimax problems with a smooth coupling term and nonsmooth convex regularizers, the resulting algorithm demonstrates significantly improved convergence properties compared to the current state of the art in the literature. For convex optimization problems with linear cone constraints, our approach yields a fully splitting primal-dual algorithm that ensures not only the convergence of iterates to a primal-dual solution, but also a \emph{last-iterate} convergence rate of $o\left( \frac{1}{k} \right)$ as $k \to +\infty$ for the objective function value, feasibility measure, and complementarity condition. This represents the most competitive theoretical result currently known for algorithms addressing this class of optimization problems. Numerical experiments are performed to illustrate the convergence behavior of Fast RFB.

Figures (6)

• Figure 4.1: The impact of the parameter $c$ on the convergence behavior of the discrete velocity, the tangent residual, the primal-dual gap, and the function values when $\alpha=3$.
• Figure 4.2: The impact of the parameter $c$ on the convergence behavior of the discrete velocity, the tangent residual, the primal-dual gap, and the function values when $\alpha=5$.
• Figure 4.3: The impact of the parameter $c$ on the convergence behavior of the discrete velocity, the tangent residual, the primal-dual gap, and the function values when $\alpha=10$.
• Figure 4.4: The impact of the parameter $c$ on the convergence behavior of the discrete velocity, the tangent residual, the primal-dual gap, and the function values when $\alpha=20$.
• Figure 4.5: A comparison of various methods in terms of discrete velocity, tangent residual, primal-dual gap, and function values for $n=1000$.

Theorems & Definitions (26)

• Proposition 2.1
• proof
• Remark 2.2
• Remark 2.3
• Lemma 2.4
• Lemma 2.5
• Lemma 2.6
• Proposition 2.7
• proof
• Theorem 2.8