On the Fundamental Limit of Stochastic Gradient Identification Algorithm Under Non-Persistent Excitation

TL;DR

It is established that strong consistency in fact holds for the entire range $0 \leq \alpha<1$, achieved through a novel algebraic framework that yields substantially sharper matrix norm bounds.

Abstract

Stochastic gradient methods are of fundamental importance in system identification and machine learning, enabling online parameter estimation for large-scale and data-streaming processes. The stochastic gradient algorithm stands as a classical identification method that has been extensively studied for decades. Under non-persistent excitation, the best known convergence result requires the condition number of the Fisher information matrix to satisfy $κ(\sum_{i=1}^n \varphi_i \varphi_i^\top) = O((\log r_n)^α)$, where $r_n = 1 + \sum_{i=1}^n \|\varphi_i\|^2$, with strong consistency guaranteed for $α\leq 1/3$ but known to fail for $α> 1$. This paper establishes that strong consistency in fact holds for the entire range $0 \leq α< 1$, achieved through a novel algebraic framework that yields substantially sharper matrix norm bounds. Our result nearly resolves the four-decade-old conjecture of Chen and Guo (1986), bridging the theoretical gap from $α\leq 1/3$ to nearly the entire feasible range.