Splitting the Forward-Backward Algorithm: A Full Characterization

TL;DR

This work provides the first partial yet comprehensive characterization of averaged frugal resolvent splittings with minimal lifting for monotone inclusions. It establishes that a broad, parameterized framework (Algorithm 1) captures all such methods and identifies precise structural conditions (causality, fixed-point encoding, and nonexpansiveness) that guarantee convergence. By introducing a lifted formulation and several practical heuristics, the paper enables distributed implementations and significant performance gains over existing schemes, as demonstrated on toy problems and portfolio optimization with decarbonization. The results have practical impact for large-scale, structured convex problems where only individual resolvent evaluations and forward steps are affordable, and they lay groundwork for future extensions to more general operator classes and compositions.

Abstract

We study frugal splitting algorithms with minimal lifting for solving monotone inclusion problems involving sums of maximal monotone and cocoercive operators. Building on a foundational result by Ryu, we fully characterize all methods that use only individual resolvent evaluations, direct evaluations of cocoercive operators, and minimal memory resources while ensuring convergence via averaged fixed-point iterations. We show that all such methods are captured by a unified framework, which includes known schemes and enables new ones with promising features. Systematic numerical experiments lead us to propose three design heuristics to achieve excellent performances in practice, yielding significant gains over existing methods.

Figures (4)

• Figure 1: How to choose the matrices $\mathcal{L}$ and $P$: Results of the experiments in Section \ref{['sec:num_testing_M']} and Section \ref{['sec:num_testing_P']}. In Figure \ref{['fig:experiment_lap']}, the blue dots correspond to $\mathcal{L}$ generated as $\mathcal{L}=MM^T$ with $M$ as in \ref{['eq:define_M']}, the red dot to the case with $\mathcal{L}$ as in \ref{['eq:define_lap']} and the black ones to those cases in which $\mathcal{L}$ is a graph laplacian.
• Figure 2: How to choose $H$ and $K$: The benefit of accounting for heterogeneity of data (Figure \ref{['fig:experiment_het']}) and the influence of the spectral norm of $W$ on the performance of Algorithm \ref{['alg:simple_splitting_FPE_introduction']} with different choices of $H$, $K$ (Figures \ref{['fig:experiment_m_obj']} and \ref{['fig:experiment_m_sca']}). Specifically, the red point in Figure \ref{['fig:experiment_m_sca']} corresponds to the red line in Figure \ref{['fig:experiment_m_obj']}. See Section \ref{['sec:num_studying_H_and_K']} for further details.
• Figure 3: Comparison between proposed methods (aGFB, SFB+) and existing schemes to solve the toy problem in Section \ref{['sec:num_data_preparation']}. See Section \ref{['sec:num_comparison_toy']} for details.
• Figure 4: Portfolio Optimization: Comparison with existing schemes and data insights. In Figure \ref{['fig:dataset']}, the blue vertical lines divide the data into four chunks, each defining one forward term. See Section \ref{['sec:num_comparison_portfolio_opt']} for further details.

Theorems & Definitions (48)

• Definition 2.1: Frugal splitting operators with minimal lifting
• Proposition 2.2
• Definition 2.3: Equivalent frugal splitting algorithms
• Proposition 2.4
• proof
• Definition 3.1: Fixed-point encoding
• Proposition 3.2
• proof
• Lemma 3.3
• proof