Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Asymptotic regularity of a generalised stochastic Halpern scheme

Nicholas Pischke, Thomas Powell

TL;DR

This work establishes a comprehensive stochastic framework for generalized Halpern–Mann iterations that incorporate two nonexpansive mappings and abstract stochastic noise. It delivers both general, quantitative rates of asymptotic regularity and fast linear (often optimal) rates in key special cases, with quadratic rates in inner product spaces for the general scheme. The authors connect these results to proof mining, derive explicit modulus-based bounds, and discuss practical variance management via minibatching and oracle complexity, highlighting potential applications to reinforcement learning and Q-learning. The contributions unify stochastic Halpern and KM-T variants, extend known deterministic results to the stochastic setting, and provide a foundation for future explorations of probabilistic fixed-point algorithms with concrete performance guarantees.

Abstract

We provide abstract, general and highly uniform rates of asymptotic regularity for a generalized stochastic Halpern-style iteration, which incorporates a second mapping in the style of a Krasnoselskii-Mann iteration. This iteration is general in two ways: First, it incorporates stochasticity completely abstractly, rather than fixing a sampling method; second, it includes as special cases stochastic versions of various schemes from the optimization literature, including Halpern's iteration as well as a Krasnoselskii-Mann iteration with Tikhonov regularization terms in the sense of Boţ, Csetnek and Meier (where this stochastic variant of the latter is considered for the first time in this paper). For these specific cases, we obtain linear rates of asymptotic regularity, matching (or improving) the currently best known rates for these iterations in stochastic optimization, and quadratic rates of asymptotic regularity are obtained in the context of inner product spaces for the general iteration. We conclude by discussing how variance can be managed in practice through sampling methods in the style of minibatching, how our convergence rates can be adapted to provide oracle complexity bounds, and by sketching how the schemes presented here can be instantiated in the context of reinforcement learning to yield novel methods for Q-learning.

Asymptotic regularity of a generalised stochastic Halpern scheme

TL;DR

This work establishes a comprehensive stochastic framework for generalized Halpern–Mann iterations that incorporate two nonexpansive mappings and abstract stochastic noise. It delivers both general, quantitative rates of asymptotic regularity and fast linear (often optimal) rates in key special cases, with quadratic rates in inner product spaces for the general scheme. The authors connect these results to proof mining, derive explicit modulus-based bounds, and discuss practical variance management via minibatching and oracle complexity, highlighting potential applications to reinforcement learning and Q-learning. The contributions unify stochastic Halpern and KM-T variants, extend known deterministic results to the stochastic setting, and provide a foundation for future explorations of probabilistic fixed-point algorithms with concrete performance guarantees.

Abstract

We provide abstract, general and highly uniform rates of asymptotic regularity for a generalized stochastic Halpern-style iteration, which incorporates a second mapping in the style of a Krasnoselskii-Mann iteration. This iteration is general in two ways: First, it incorporates stochasticity completely abstractly, rather than fixing a sampling method; second, it includes as special cases stochastic versions of various schemes from the optimization literature, including Halpern's iteration as well as a Krasnoselskii-Mann iteration with Tikhonov regularization terms in the sense of Boţ, Csetnek and Meier (where this stochastic variant of the latter is considered for the first time in this paper). For these specific cases, we obtain linear rates of asymptotic regularity, matching (or improving) the currently best known rates for these iterations in stochastic optimization, and quadratic rates of asymptotic regularity are obtained in the context of inner product spaces for the general iteration. We conclude by discussing how variance can be managed in practice through sampling methods in the style of minibatching, how our convergence rates can be adapted to provide oracle complexity bounds, and by sketching how the schemes presented here can be instantiated in the context of reinforcement learning to yield novel methods for Q-learning.

Paper Structure

This paper contains 21 sections, 30 theorems, 155 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Background and motivation
  3. Overview of the main results
  4. General (quantitative) asymptotic regularity results
  5. Fast rates of asymptotic regularity
  6. Future applications and developments
  7. This paper in connection with the proof mining program
  8. Preliminaries and basic lemmas
  9. Quantitative asymptotic regularity for the generalized stochastic Halpern scheme
  10. Basic results and rates of asymptotic regularity
  11. Asymptotic regularity relative to the mappings
  12. Special cases of the Halpern iteration and the Krasnoselskii-Mann iteration with Tikhonov regularization terms
  13. The general case
  14. Fast rates of asymptotic regularity
  15. General results on linear rates
  16. ...and 6 more sections

Key Result

Lemma 2.1

Suppose that $(s_n),(c_n)\subseteq [0,\infty)$ as well as $(a_n)\subseteq [0,1]$ and $(b_n)\subseteq\mathbb{R}$ satisfy for all $n\in\mathbb{N}$ where $\sum_{n=0}^\infty a_n=\infty$, $\limsup b_n\leq 0$ and $\sum_{n=0}^\infty c_n<\infty$. Then $s_n\to 0$.

Figures (1)

  • Figure 1: Relationships of the stochastic Halpern-type schemes and their deterministic variants

Theorems & Definitions (65)

  • Lemma 2.1: Xu2002
  • Lemma 2.2: essentially KL2012LP2021
  • Lemma 2.3
  • proof
  • Lemma 2.4
  • proof
  • Lemma 3.1: essentially LP2024
  • proof
  • Theorem 3.2
  • proof
  • ...and 55 more