Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An error bound-based convergence analysis framework for a class of randomized algorithms

Zhichun Yang, Li Jiang, Tianxiang Liu, Man-Chung Yue

Abstract

Existing error-bound-based analyses for stochastic algorithms that exhibit certain descent properties, such as randomized coordinate descent and randomized projection methods, are often limited in scope and typically lead to overly conservative convergence guarantees. To address this gap, we develop an abstract framework for analyzing such stochastic algorithms based on new unified error bound (UEB) conditions. The proposed UEB conditions subsume many common error bound- and Kurdyka--Łojasiewicz-type conditions used in existing studies of algorithms for optimization, convex feasibility, and common fixed point problems. Under the global UEB condition, we establish non-asymptotic in-expectation and asymptotic almost-sure convergence rates for the stochastic algorithms in our framework. Under the local UEB condition, we also show asymptotic almost sure convergence rates. We demonstrate the strength and versatility of our framework through two applications. For the common fixed point problem, we provide comprehensive convergence guarantees for the randomized alternating Krasnoselskii-Mann method under Hölderian error bound conditions. Furthermore, for unconstrained minimization of smooth definable functions, we establish novel convergence guarantees for the randomized subspace descent method, an algorithm subsuming both randomized coordinate and block coordinate descent.

An error bound-based convergence analysis framework for a class of randomized algorithms

Abstract

Existing error-bound-based analyses for stochastic algorithms that exhibit certain descent properties, such as randomized coordinate descent and randomized projection methods, are often limited in scope and typically lead to overly conservative convergence guarantees. To address this gap, we develop an abstract framework for analyzing such stochastic algorithms based on new unified error bound (UEB) conditions. The proposed UEB conditions subsume many common error bound- and Kurdyka--Łojasiewicz-type conditions used in existing studies of algorithms for optimization, convex feasibility, and common fixed point problems. Under the global UEB condition, we establish non-asymptotic in-expectation and asymptotic almost-sure convergence rates for the stochastic algorithms in our framework. Under the local UEB condition, we also show asymptotic almost sure convergence rates. We demonstrate the strength and versatility of our framework through two applications. For the common fixed point problem, we provide comprehensive convergence guarantees for the randomized alternating Krasnoselskii-Mann method under Hölderian error bound conditions. Furthermore, for unconstrained minimization of smooth definable functions, we establish novel convergence guarantees for the randomized subspace descent method, an algorithm subsuming both randomized coordinate and block coordinate descent.
Paper Structure (16 sections, 25 theorems, 144 equations)

This paper contains 16 sections, 25 theorems, 144 equations.

Table of Contents

  1. Introduction
  2. Notation
  3. Problem Setup and Preliminaries
  4. A Class of Structured Monotone Randomized Algorithms
  5. Auxiliaries
  6. Convergence Analysis under Global Error Bounds
  7. The case when $\phi_{S}$ is convex
  8. The case with a general $\phi_S$
  9. An Apparatus for In-Expectation Convergence Rate Analysis
  10. An Apparatus for Almost-Sure Convergence Rate Analysis
  11. Apparatuses for Transferring Convergence Rates of $h(x^k)$ to $x^k$
  12. Proof of Theorem \ref{['th_general_exp_rate1']}
  13. Convergence Analysis under Local Error Bounds
  14. Applications
  15. Application to RKMM
  16. ...and 1 more sections

Key Result

Lemma 1

For the function $\phi$ defined in Definition def:inverse-smoothing, the following properties hold:

Theorems & Definitions (56)

  • Example 1
  • Example 2
  • Definition 1: Inverse smoothing and desingularization functions
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Theorem 1
  • ...and 46 more