Recursive Algorithms for Sparse Parameter Identification of Multivariate Stochastic Systems with Non-stationary Observations
Yanxin Fu, Wenxiao Zhao
TL;DR
This work develops recursive algorithms for online sparse parameter identification in multivariate stochastic systems with non-stationary observations by introducing an auxiliary variable matrix $\Xi$ into a weighted $L_1$ regularization framework. The problem is solved via online alternating optimization, yielding explicit recursive updates for $\Theta$ and $\Xi$ with a soft-thresholding step and a dynamically adjusted penalty $\gamma_N(s,t)$. Theoretical results establish set convergence (exact sparsity pattern recovery) and parameter convergence (consistent nonzero entries) under non-stationary excitation, with weaker conditions than prior approaches. Numerical experiments on a SISO ARX-like system and the Lorenz system demonstrate superior sparsity recovery and parameter accuracy compared to several online methods and SINDy, validating the practical impact for online sparse system identification with evolving data. The approach offers a principled, online solution to sparsity inference in control-relevant settings where observations are non-stationary and data arrive sequentially.
Abstract
The classical sparse parameter identification methods are usually based on the iterative basis selection such as greedy algorithms, or the numerical optimization of regularized cost functions such as LASSO and Bayesian posterior probability distribution, etc., which, however, are not suitable for online sparsity inference when data arrive sequentially. This paper presents recursive algorithms for sparse parameter identification of multivariate stochastic systems with non-stationary observations. First, a new bivariate criterion function is presented by introducing an auxiliary variable matrix into a weighted $L_1$ regularization criterion. The new criterion function is subsequently decomposed into two solvable subproblems via alternating optimization of the two variable matrices, for which the optimizers can be explicitly formulated into recursive equations. Second, under the non-stationary and non-persistent excitation conditions on the systems, theoretical properties of the recursive algorithms are established. That is, the estimates are proved to be with (i) set convergence, i.e., the accurate estimation of the sparse index set of the unknown parameter matrix, and (ii) parameter convergence, i.e., the consistent estimation for values of the non-zero elements of the unknown parameter matrix. Finally, numerical examples are given to support the theoretical analysis.