Asymptotic Efficiency Analysis of the Recursive Least-Squares Algorithm for ARX Systems Without Projection
Xingrui Liu, Jieming Ke, Yanlong Zhao
TL;DR
This work removes the traditional projection-based compact-set assumption in the asymptotic analysis of recursive least-squares for ARX systems by developing a tail-probability framework that ties estimation error behavior to the inverse covariance of regression vectors under quasi-stationary inputs. By employing a combinatorial approach, the authors derive moment-based conditions that yield three main results: (i) asymptotic normality with CRLB-consistent variance under $8$-th moment bounds, (ii) covariance convergence to the CRLB under $20$-th moment bounds, and (iii) $L^{\gamma/2}$ convergence with rate $O(1/k^{\gamma/4})$ under $4\gamma$-th moment bounds. The key conclusion is that the RLS algorithm can be asymptotically optimal for ARX identification even without projection, expanding the scope of RLS applicability in practice. The findings hinge on linking tail probabilities of the estimation error to the growth of the inverse covariance and require bounded higher-order moments of the input/output signals. This work has potential impact on real-time adaptive control and system identification where projection steps are undesirable or infeasible.
Abstract
This paper investigates the optimality analysis of the recursive least-squares (RLS) algorithm for autoregressive systems with exogenous inputs (ARX systems). A key challenge in analyzing is managing the potential unboundedness of the parameter estimates, which may diverge to infinity. Previous approaches addressed this issue by assuming that both the true parameter and the RLS estimates remain confined within a known compact set, thereby ensuring uniform boundedness throughout the analysis. In contrast, we propose a new analytical framework that eliminates the need for such a boundness assumption. Specifically, we establish a quantitative relationship between the bounded moment conditions of quasi-stationary input/output signals and the convergence rate of the tail probability of the RLS estimation error. Based on this technique, we prove that when system inputs/outputs have bounded twentieth-order moments, the RLS algorithm achieves asymptotic normality and the covariance matrix of the RLS algorithm converges to the Cramér-Rao lower bound (CRLB), confirming its asymptotic efficiency. These results demonstrate that the RLS algorithm is an asymptotically optimal identification algorithm for ARX systems, even without the projection operators to ensure that parameter estimates reside within a prior known compact set.