Randomized subspace correction methods for convex optimization
Boou Jiang, Jongho Park, Jinchao Xu
TL;DR
The paper develops an abstract framework for randomized subspace correction methods applied to convex optimization, unifying domain decomposition, multigrid, and block coordinate descent under inexact local solvers and general space decompositions. It establishes convergence theorems across general, sharp, and strongly convex problems by relating expected energy descent to a computable quantity and stable decomposition constants, then shows how linear problems, block coordinate descent, and operator splitting are recovered as special cases. The results extend existing analyses to nonsmooth settings, weaker smoothness, and broader local solvers, enabling applicability to nonlinear PDEs, variational inequalities, TV minimization, and large-scale statistical models. The framework provides concrete, verifiable rates and descent properties, offering a versatile tool for designing and analyzing randomized iterative schemes in computation and data science.
Abstract
This paper introduces an abstract framework for randomized subspace correction methods for convex optimization, which unifies and generalizes a broad class of existing algorithms, including domain decomposition, multigrid, and block coordinate descent methods. We provide a convergence rate analysis ranging from minimal assumptions to more practical settings, such as sharpness and strong convexity. While most existing studies on block coordinate descent methods focus on nonoverlapping decompositions and smooth or strongly convex problems, our framework extends to more general settings involving arbitrary space decompositions, inexact local solvers, and problems with weaker smoothness or convexity assumptions. The proposed framework is broadly applicable to convex optimization problems arising in areas such as nonlinear partial differential equations, imaging, and data science.