Relaxed and inertial nonlinear Forward-Backward algorithm
Juan José Maulén, Fernando Roldán, Cristian Vega
TL;DR
The paper addresses solving monotone inclusions of the form 0 ∈ Ax + Cx by presenting an inertial and relaxed extension of the nonlinear Forward-Backward (NFB) algorithm within a warped resolvent framework. It shows weak convergence for both nondecreasing and decreasing inertial sequences, and even linear convergence when A is strongly monotone, while unifying and extending several known algorithms (FB, FBF, FBHF, Chambolle–Pock, Condat–Vũ, FPDHF) as special cases. New contributions include inertial-relaxed variants of FBHF and FPDHF and the novel consideration of decreasing inertial parameters, along with an initialization strategy to aid practical use. Numerical experiments on affine-constrained optimization and image restoration illustrate potential convergence acceleration from decreasing inertia, highlighting the method’s practical relevance and the need for principled parameter selection.
Abstract
The Nonlinear Forward-Backward (NFB) algorithm, also known as warped resolvent iterations, is a splitting method for finding zeros of sums of monotone operators. In particular cases, NFB reduces to well-known algorithms such as Forward-Backward, Forward-Backward-Forward, Chambolle--Pock, and Condat--Vũ. Therefore, NFB can be used to solve monotone inclusions involving sums of maximally monotone, cocoercive, monotone and Lipschitz operators as well as linear compositions terms. In this article, we study the weak and strong (linear) convergence of NFB with inertial and relaxation steps. Our results recover known convergence guarantees for the aforementioned methods when extended with inertial and relaxation terms. Additionally, we establish the convergence of inertial and relaxed variants of the Forward-Backward-Half-Forward and Forward-Primal--Dual-Half-Forward algorithms, which, to the best of our knowledge, are new contributions. We consider both nondecreasing and decreasing sequences of inertial parameters, the latter being a novel approach in the context of inertial algorithms. To evaluate the performance of these strategies, we present numerical experiments on optimization problems with affine constraints and on image restoration tasks. Our results show that decreasing inertial sequences can accelerate the numerical convergence of the algorithms.