Error bounds of constant gain least-mean-squares algorithms
Chang Liu, Antwan D. Clark
TL;DR
This work addresses the lack of a formal mean-squared convergence guarantee for constant-gain LMS algorithms by deriving sufficient upper bounds on the constant gain $a$ that ensure a bounded mean-squared error (MSE) for a general design vector. Using a Lyapunov-based analysis with a positive definite matrix $P$ and a contraction parameter $\chi$, the authors establish a matrix inequality involving the fourth-order moment of the design vector, yielding explicit bounds on $\mathbb{E}[\|\hat{\boldsymbol{\theta}}_k-\boldsymbol{\theta}^*\|_2^2]$ and its asymptotic limit $\lim_{k\to\infty}\Pi_k$. A corollary for $P=I$ provides a simpler criterion, while numerical examples (including synthetic and practical reed data) demonstrate that the proposed bounds can be tighter and applicable to linearly dependent design vectors, often outperforming existing criteria such as those in Widrow (1985) and Zhu-Spall (2015). The results offer practical guidance for selecting constant gains in trajectory tracking and temporal-parameter estimation, and point to future work on dependence between $\boldsymbol{h}_k$ and $\epsilon_k$, time-varying parameters, and convergence rate analysis.
Abstract
Constant gain least-mean-squares (LMS) algorithms have a wide range of applications in trajectory tracking problems, but the formal convergence of LMS in mean square is not yet fully established. This work provides an upper bound on the constant gain that guarantees a bounded mean-squared error of LMS for a general design vector. These results highlight the role of the fourth-order moment of the design vector. Numerical examples demonstrate the applicability of this upper bound in setting a constant gain in LMS, while existing criteria may fail. We also provide the associated error bound, which can be applied to design vectors with linearly dependent elements.