Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic Recursive Inclusions under Biased Perturbations: An Input-to-State Stability Perspective

Anik Kumar Paul, Karthik Shenoy, Arun D. Mahindrakar

TL;DR

This work analyzes stochastic recursive inclusions with nonzero persistent bias through the input-to-state stability (ISS) lens of the associated differential inclusion $\dot{x}(t)\in H(x(t))+\bar{B}(0,\epsilon)$. By coupling ISS with Robbins–Monro–type step-sizes and martingale noise, it shows that, when iterates are almost surely bounded, the discrete recursion converges to a neighborhood of equilibria whose radius scales with the bias. A verifiable boundedness condition is provided for the case of a single-valued globally Lipschitz $H$, and the results extend to zeroth-order and biased stochastic gradient methods in nonconvex, convex, and constrained settings. Simulations illustrate the bias–variance tradeoff and validate convergence to neighborhoods controlled by $\epsilon$ under standard PL and Lipschitz assumptions. Overall, the paper offers a unified ISS-based framework for almost-sure convergence of biased stochastic approximation schemes across diverse applications.

Abstract

This paper investigates the asymptotic behavior of stochastic recursive inclusions in the presence of non-zero, non-diminishing bias, a setting that frequently arises in zeroth-order optimization, stochastic approximation with iterate-dependent noise, and distributed learning with adversarial agents. The analysis is conducted through the lens of input-to-state stability of an associated differential inclusion, which serves as the continuous-time limit of the discrete recursion. We first establish that if the limiting differential inclusion is input-to-state stable and the iterates remain almost surely bounded, then the iterates converge almost surely to the neighborhood of desired equilibrium. We then provide a verifiable sufficient condition for almost sure boundedness by assuming that the underlying operator is single-valued and globally Lipschitz. Finally, we show that several zeroth-order variants of stochastic gradient naturally fit within this framework, and we demonstrate their input-to-state stability under standard conditions. Overall, the results provide a unified theoretical foundation for studying almost sure convergence of biased stochastic approximation schemes through the Input to State stability theory of differential inclusions.

Stochastic Recursive Inclusions under Biased Perturbations: An Input-to-State Stability Perspective

TL;DR

This work analyzes stochastic recursive inclusions with nonzero persistent bias through the input-to-state stability (ISS) lens of the associated differential inclusion . By coupling ISS with Robbins–Monro–type step-sizes and martingale noise, it shows that, when iterates are almost surely bounded, the discrete recursion converges to a neighborhood of equilibria whose radius scales with the bias. A verifiable boundedness condition is provided for the case of a single-valued globally Lipschitz , and the results extend to zeroth-order and biased stochastic gradient methods in nonconvex, convex, and constrained settings. Simulations illustrate the bias–variance tradeoff and validate convergence to neighborhoods controlled by under standard PL and Lipschitz assumptions. Overall, the paper offers a unified ISS-based framework for almost-sure convergence of biased stochastic approximation schemes across diverse applications.

Abstract

This paper investigates the asymptotic behavior of stochastic recursive inclusions in the presence of non-zero, non-diminishing bias, a setting that frequently arises in zeroth-order optimization, stochastic approximation with iterate-dependent noise, and distributed learning with adversarial agents. The analysis is conducted through the lens of input-to-state stability of an associated differential inclusion, which serves as the continuous-time limit of the discrete recursion. We first establish that if the limiting differential inclusion is input-to-state stable and the iterates remain almost surely bounded, then the iterates converge almost surely to the neighborhood of desired equilibrium. We then provide a verifiable sufficient condition for almost sure boundedness by assuming that the underlying operator is single-valued and globally Lipschitz. Finally, we show that several zeroth-order variants of stochastic gradient naturally fit within this framework, and we demonstrate their input-to-state stability under standard conditions. Overall, the results provide a unified theoretical foundation for studying almost sure convergence of biased stochastic approximation schemes through the Input to State stability theory of differential inclusions.
Paper Structure (9 sections, 8 theorems, 96 equations, 2 figures)

This paper contains 9 sections, 8 theorems, 96 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Notation and Math Preliminaries
  3. Asymptotic Convergence Analysis
  4. Almost Sure Boundedness
  5. Leveraging Theorem \ref{['thm:eps_convergence']} for Tackling Non-Convex Unconstrained Challenges
  6. Leveraging Theorem \ref{['thm:eps_convergence']} to Address Convex Constrained Optimization Challenges
  7. Simulation Results
  8. Conclusion
  9. Proof of Proposition \ref{['prop:finite_horizon_bound']}

Key Result

Proposition 1

Suppose Assumptions ISS, step-size, Mar, and noise hold, and further assume that ${x_n}$ is bounded almost surely. Then, the continuous-time interpolation $\Bar{X}(t)$ of ${x_n}$ is an asymptotic pseudo-trajectory (APT) of the differential inclusion That is, where $x_t(s)$ denotes the solution of Perturbed with the initial condition $x_t(0) = \Bar{X}(t)$.

Figures (2)

  • Figure 1: The top plot depicts $|f_1(x_n)-f_1^*|$, for $\lambda=0.05,\;0.1,\;1$ and the bottom plot depicts the same for $\lambda=0.0005$. Moreover, $|f_1(x_{100000})-f_1^*|=73.4935, 0.0263, 0.0201 \;\mathrm{and}\;0.0316$ for $\lambda=0.0005,0.05,0.1\;\mathrm{and}\;1$, respectively.
  • Figure 2: The top plot depicts $|f_2(x_n)-f_2^*|$, for $\lambda=0.05,\;0.1,\;1$ and the bottom plot depicts the same for $\lambda=0.0005$. Moreover, $|f_2(x_{100000})-f_2^*|=0.0064,\; 0.0068,\; 0.2545$ for $\lambda=0.05,\;0.1\;\mathrm{and}\;1$, respectively.

Theorems & Definitions (16)

  • Proposition 1
  • Theorem 1: Almost Sure Convergence
  • proof
  • Remark 1
  • Proposition 2
  • Lemma 1
  • proof
  • Theorem 2
  • proof
  • Proposition 3
  • ...and 6 more