From Halpern's Fixed-Point Iterations to Nesterov's Accelerated Interpretations for Root-Finding Problems
Quoc Tran-Dinh
TL;DR
The paper advances a unified view of acceleration for fixed-point and monotone-inclusion problems by establishing a rigorous equivalence between Halpern's fixed-point iteration and Nesterov's accelerated schemes under a $\tfrac{1}{L}$-co-coercive operator. It then extends this equivalence to derive new Nesterov-accelerated variants for proximal-point, forward-backward, and three-operator splitting, as well as for extra-anchored gradient methods, even when co-coerciveness is relaxed to mere monotonicity and Lipschitz continuity. Through Lyapunov/energy-function analyses, the authors obtain $O(1/k^2)$-type and $o(1/k^2)$-type convergence results and prove convergence of all iterates in many settings, complemented by corollaries for specific parameter choices. Numerical experiments on linear regression and minimax saddle problems corroborate the theoretical rates and demonstrate practical speedups of the accelerated schemes in both co-coercive and non-coercive scenarios.
Abstract
We derive an equivalent form of Halpern's fixed-point iteration scheme for solving a co-coercive equation (also called a root-finding problem), which can be viewed as a Nesterov's accelerated interpretation. We show that one method is equivalent to another via a simple transformation, leading to a straightforward convergence proof for Nesterov's accelerated scheme. Alternatively, we directly establish convergence rates of Nesterov's accelerated variant, and as a consequence, we obtain a new convergence rate of Halpern's fixed-point iteration. Next, we apply our results to different methods to solve monotone inclusions, where our convergence guarantees are applied. Since the gradient/forward scheme requires the co-coerciveness of the underlying operator, we derive new Nesterov's accelerated variants for both recent extra-anchored gradient and past-extra anchored gradient methods in the literature. These variants alleviate the co-coerciveness condition by only assuming the monotonicity and Lipschitz continuity of the underlying operator. Interestingly, our new Nesterov's accelerated interpretation of the past-extra anchored gradient method involves two past-iterate correction terms. This formulation is expected to guide us developing new Nesterov's accelerated methods for minimax problems and their continuous views without co-coericiveness. We test our theoretical results on two numerical examples, where the actual convergence rates match well the theoretical ones up to a constant factor.