Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling

TL;DR

The paper tackles the challenge of modelling nonlinear dynamic systems with uncertain dynamic orders by coupling LASSO-based order-variable selection with a residual modelling stage implemented as an $L_2$-regularized Recurrent Stochastic Configuration Network (RSCN). The core method first extracts significant order variables via LASSO, yielding $Y_{LASSO}$ and residual $\hat{Y}=T-Y_{LASSO}$, and then trains an online, projection-updated RSCN to approximate $\hat{Y}$ with a compact reservoir. A theoretical universal approximation analysis demonstrates that the hybrid framework retains the echo state property and guarantees convergence of the online weight updates. Empirical results on nonlinear system identification and two industrial predictive tasks show the proposed LASSO-RSCN-L2 approach achieving superior accuracy and robustness with smaller reservoirs, indicating strong potential for real-time, data-driven nonlinear dynamics modelling in engineering applications.

Abstract

Recurrent stochastic configuration networks (RSCNs) have shown great potential in modelling nonlinear dynamic systems with uncertainties. This paper presents an RSCN with hybrid regularization to enhance both the learning capacity and generalization performance of the network. Given a set of temporal data, the well-known least absolute shrinkage and selection operator (LASSO) is employed to identify the significant order variables. Subsequently, an improved RSCN with L2 regularization is introduced to approximate the residuals between the output of the target plant and the LASSO model. The output weights are updated in real-time through a projection algorithm, facilitating a rapid response to dynamic changes within the system. A theoretical analysis of the universal approximation property is provided, contributing to the understanding of the network's effectiveness in representing various complex nonlinear functions. Experimental results from a nonlinear system identification problem and two industrial predictive tasks demonstrate that the proposed method outperforms other models across all testing datasets.

Figures (11)

• Figure 1: Architectures of the hybrid regularized RSCN.
• Figure 2: The distribution of feature coefficients corresponding to each order variable for the nonlinear system identification task.
• Figure 3: The prediction fitting curves and error values of different models for the nonlinear system identification task.
• Figure 4: The testing NRMSE surface map of LASSO-RSCN-L2 with different regularization coefficient $C$ and reservoir size on the nonlinear system identification task.
• Figure 5: Flowchart of debutanizer column process.