On Inhomogeneous Infinite Products of Stochastic Matrices and Applications

TL;DR

The paper analyzes the convergence properties of inhomogeneous infinite products of stochastic matrices (IPSMs) and their limiting absolute probability sequences, establishing strong ergodicity under weak, time-varying graph assumptions and deriving a near-exponential convergence rate toward $\pi(s)$. It further characterizes the limit of the absolute probability sequence when the matrices approach the identity at a rate $O(1/\ln k)$, showing convergence to the uniform distribution with a strong $\sum_{k=1}^\infty \frac{1}{k}\left|\pi_i(k)-\frac{1}{n}\right|<\infty$ bound. Leveraging these results, the authors introduce a decentralized projected subgradient method with graph-informed gradient stretches (UDPSG) for time-varying unbalanced directed graphs, proving convergence to global minima for convex objectives and extending to non-convex PL-satisfying objectives. Numerical simulations corroborate the theory and indicate performance gains over standard decentralized projected subgradient methods, underscoring the approach’s relevance for scalable, robust decentralized optimization in dynamic networks.

Abstract

With the growth of magnitude of multi-agent networks, distributed optimization holds considerable significance within complex systems. Convergence, a pivotal goal in this domain, is contingent upon the analysis of infinite products of stochastic matrices (IPSMs). In this work, convergence properties of inhomogeneous IPSMs are investigated. The convergence rate of inhomogeneous IPSMs towards an absolute probability sequence $π$ is derived. We also show that the convergence rate is nearly exponential, which coincides with existing results on ergodic chains. The methodology employed relies on delineating the interrelations among Sarymsakov matrices, scrambling matrices, and positive-column matrices. Based on the theoretical results on inhomogeneous IPSMs, we propose a decentralized projected subgradient method for time-varying multi-agent systems with graph-related stretches in (sub)gradient descent directions. The convergence of the proposed method is established for convex objective functions, and extended to non-convex objectives that satisfy Polyak-Lojasiewicz conditions. To corroborate the theoretical findings, we conduct numerical simulations, aligning the outcomes with the established theoretical framework.

Figures (6)

• Figure 1: The evolution of inhomogeneous IPSM.
• Figure 2: Average consensus errors in the case of local convex objective functions.
• Figure 3: Value of the global convex objective functions.
• Figure 4: Value of the aggregated non-convex functions.
• Figure 5: Average consensus errors with respect to different methods.

Theorems & Definitions (35)

• Definition 1: Sarymsakov Matrix 1961SarymsakovMatrix
• Definition 2: Scrambling Matrix 1963ProdSIAMatrices
• Remark
• Lemma 1
• proof
• Corollary 1
• proof
• Lemma 2
• proof
• Lemma 3