Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Systems-Theoretic View on the Convergence of Algorithms under Disturbances

Guner Dilsad Er, Sebastian Trimpe, Michael Muehlebach

TL;DR

This work treats iterative algorithms as dynamical systems embedded in noisy environments and derives rate-preserving convergence bounds under disturbances using converse Lyapunov theory. It unifies deterministic and stochastic disturbance analyses, connects to ISS and small-gain concepts, and demonstrates practical applications in distributed optimization, generalization, and privacy-preserving learning. The results recover and extend classical bounds (e.g., Hardt 2016) while providing a general, Lyapunov-based toolkit for robustness in learning and interconnected systems. Overall, it offers a principled framework for assessing and designing algorithms that operate reliably under noise, delays, and environmental feedback. This framework has broad implications for the design of robust, privacy-aware, and communication-efficient learning systems.

Abstract

Algorithms increasingly operate within complex physical, social, and engineering systems where they are exposed to disturbances, noise, and interconnections with other dynamical systems. This article extends known convergence guarantees of an algorithm operating in isolation (i.e., without disturbances) and systematically derives stability bounds and convergence rates in the presence of such disturbances. By leveraging converse Lyapunov theorems, we derive key inequalities that quantify the impact of disturbances. We further demonstrate how our result can be utilized to assess the effects of disturbances on algorithmic performance in a wide variety of applications, including communication constraints in distributed learning, sensitivity in machine learning generalization, and intentional noise injection for privacy. This underpins the role of our result as a unifying tool for algorithm analysis in the presence of noise, disturbances, and interconnections with other dynamical systems.

A Systems-Theoretic View on the Convergence of Algorithms under Disturbances

TL;DR

This work treats iterative algorithms as dynamical systems embedded in noisy environments and derives rate-preserving convergence bounds under disturbances using converse Lyapunov theory. It unifies deterministic and stochastic disturbance analyses, connects to ISS and small-gain concepts, and demonstrates practical applications in distributed optimization, generalization, and privacy-preserving learning. The results recover and extend classical bounds (e.g., Hardt 2016) while providing a general, Lyapunov-based toolkit for robustness in learning and interconnected systems. Overall, it offers a principled framework for assessing and designing algorithms that operate reliably under noise, delays, and environmental feedback. This framework has broad implications for the design of robust, privacy-aware, and communication-efficient learning systems.

Abstract

Algorithms increasingly operate within complex physical, social, and engineering systems where they are exposed to disturbances, noise, and interconnections with other dynamical systems. This article extends known convergence guarantees of an algorithm operating in isolation (i.e., without disturbances) and systematically derives stability bounds and convergence rates in the presence of such disturbances. By leveraging converse Lyapunov theorems, we derive key inequalities that quantify the impact of disturbances. We further demonstrate how our result can be utilized to assess the effects of disturbances on algorithmic performance in a wide variety of applications, including communication constraints in distributed learning, sensitivity in machine learning generalization, and intentional noise injection for privacy. This underpins the role of our result as a unifying tool for algorithm analysis in the presence of noise, disturbances, and interconnections with other dynamical systems.

Paper Structure

This paper contains 16 sections, 7 theorems, 80 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries and Problem Setup
  4. Definitions
  5. Nominal Convergence of Algorithms
  6. Main Results
  7. Extension to Zero-Mean Stochastic Disturbances
  8. Relation to Input-to-State Stability
  9. Relation to Interconnected Systems and Small Gain Theorem
  10. Proof of Theorem \ref{['thm:mainthm']}
  11. Applications to Learning Algorithms
  12. Example: Distributed Optimization under Communication Constraints
  13. Example: Generalization and Algorithmic Performance
  14. Example: Privacy-Preserving Learning Mechanisms
  15. Conclusions
  16. ...and 1 more sections

Key Result

Theorem 3.1

\newlabelthm:mainthm0 Let the unperturbed algorithm be represented by the nominal dynamics in eqn:nominal_system_dynamics and satisfy Assumptions ass:uni_Lipschitz_f, ass:bounded_effect_g, and ass:convergence. Then, there exists a constant $L_V>0$, such that the following bound holds for the pertu for all $z_0 \in \mathcal{S}$, where $S\subset \mathbb{R}^d$ is compact, $z_k$ denotes the state of

Figures (1)

  • Figure 1: Illustration of an interconnected system

Theorems & Definitions (17)

  • Definition 2.3
  • Example 2.5
  • Theorem 3.1
  • Remark 3.2
  • Theorem 3.6
  • Proof 1
  • Corollary 3.7
  • Proof 2
  • Theorem 4.1
  • Proof 3
  • ...and 7 more