Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Recurrent Stochastic Configuration Networks for Modelling Nonlinear Dynamic Systems

Gang Dang, Dianhui Wang

TL;DR

Experimental results demonstrate that the proposed DeepRSCN outperforms other state-of-the-art methods with sound learning and generalization performance, and can dynamically adjust the network parameters in response to real-time data, enabling them to adapt to changing operational conditions.

Abstract

Deep learning techniques have shown promise in many domain applications. This paper proposes a novel deep reservoir computing framework, termed deep recurrent stochastic configuration network (DeepRSCN) for modelling nonlinear dynamic systems. DeepRSCNs are incrementally constructed, with all reservoir nodes directly linked to the final output. The random parameters are assigned in the light of a supervisory mechanism, ensuring the universal approximation property of the built model. The output weights are updated online using the projection algorithm to handle the unknown dynamics. Given a set of training samples, DeepRSCNs can quickly generate learning representations, which consist of random basis functions with cascaded input and readout weights. Experimental results over a time series prediction, a nonlinear system identification problem, and two industrial data predictive analyses demonstrate that the proposed DeepRSCN outperforms the single-layer network in terms of modelling efficiency, learning capability, and generalization performance.

Deep Recurrent Stochastic Configuration Networks for Modelling Nonlinear Dynamic Systems

TL;DR

Experimental results demonstrate that the proposed DeepRSCN outperforms other state-of-the-art methods with sound learning and generalization performance, and can dynamically adjust the network parameters in response to real-time data, enabling them to adapt to changing operational conditions.

Abstract

Deep learning techniques have shown promise in many domain applications. This paper proposes a novel deep reservoir computing framework, termed deep recurrent stochastic configuration network (DeepRSCN) for modelling nonlinear dynamic systems. DeepRSCNs are incrementally constructed, with all reservoir nodes directly linked to the final output. The random parameters are assigned in the light of a supervisory mechanism, ensuring the universal approximation property of the built model. The output weights are updated online using the projection algorithm to handle the unknown dynamics. Given a set of training samples, DeepRSCNs can quickly generate learning representations, which consist of random basis functions with cascaded input and readout weights. Experimental results over a time series prediction, a nonlinear system identification problem, and two industrial data predictive analyses demonstrate that the proposed DeepRSCN outperforms the single-layer network in terms of modelling efficiency, learning capability, and generalization performance.

Paper Structure

This paper contains 15 sections, 34 equations, 13 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Technical support
  3. Stochastic configuration networks
  4. Echo state network and its deep version
  5. Deep recurrent stochastic configuration networks
  6. Algorithm description
  7. Universal approximation property
  8. Online update of the output weights
  9. Experiment results
  10. Mackey–Glass system (MGS)
  11. Nonlinear system identification
  12. Soft sensing of the butane concentration in the debutanizer column process
  13. Short-term power load forecasting
  14. Discussions
  15. Conclusion

Figures (13)

  • Figure 1: Architecture of the basic RSCN.
  • Figure 2: Architecture of the basic DeepRSCN with $S$ reservoir layers.
  • Figure 3: Validation performance of different models on the MG task.
  • Figure 4: Training time comparison between the RSC-based frameworks with different reservoir sizes on the three MG tasks. (The reservoir size represents the total number of nodes across all layers, with an even number of nodes in each layer.)
  • Figure 5: The correlation between reservoir outputs and target outputs across varying model depths on the nonlinear system identification task.
  • ...and 8 more figures