Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Encoder initialisation methods in the model augmentation setting

J. H. Hoekstra, B. Györök, R. Töth, M. Schoukens

TL;DR

The paper addresses the challenge of initializing encoders in model-augmented nonlinear system identification to improve convergence and noise robustness. It introduces model-based and data-based encoder initialisation strategies that leverage baseline models, including LTI and local-linear NL generalisations, and extends them to noise-aware formulations. Through a nonlinear 2-DOF mass-spring-damper case, the authors show that baseline-informed initialisation accelerates training and yields comparable final accuracy to random initialisation, while offering improved interpretability via the augmentation structure. The work advances practical NL-SI by enabling more reliable and faster learning of encoder-based state initialization in control-relevant, physics-informed settings, with implications for online learning and interpretable model augmentation.

Abstract

Nonlinear system identification (NL-SI) has proven to be effective in obtaining accurate models for highly complex systems. Recent encoder-based methods for artificial neural network state-space (ANN-SS) models have shown state-of-the-art performance with improved computational efficiency, where the encoder is used to estimate the initial state allowing for batch optimisation methods. To address the lack of interpretability of these black-box ANN models, model augmentation approaches can be used. These combine prior available baseline models with the ANN learning components, resulting in faster convergence and more interpretable models. The combination of the encoder-based method with model augmentation has shown potential. Thus far, however, the encoder has still been treated as a black-box function in the overall estimation process, while additional information in the form of the baseline model is available to predict the model state from past input-output data. In this paper, we propose novel encoder initialisation approaches based on the available baseline model, resulting in improved noise robustness and faster convergence compared to black-box initialisation. The performance of these initialisation methods is demonstrated on a mass-spring-damper system.

Encoder initialisation methods in the model augmentation setting

TL;DR

The paper addresses the challenge of initializing encoders in model-augmented nonlinear system identification to improve convergence and noise robustness. It introduces model-based and data-based encoder initialisation strategies that leverage baseline models, including LTI and local-linear NL generalisations, and extends them to noise-aware formulations. Through a nonlinear 2-DOF mass-spring-damper case, the authors show that baseline-informed initialisation accelerates training and yields comparable final accuracy to random initialisation, while offering improved interpretability via the augmentation structure. The work advances practical NL-SI by enabling more reliable and faster learning of encoder-based state initialization in control-relevant, physics-informed settings, with implications for online learning and interpretable model augmentation.

Abstract

Nonlinear system identification (NL-SI) has proven to be effective in obtaining accurate models for highly complex systems. Recent encoder-based methods for artificial neural network state-space (ANN-SS) models have shown state-of-the-art performance with improved computational efficiency, where the encoder is used to estimate the initial state allowing for batch optimisation methods. To address the lack of interpretability of these black-box ANN models, model augmentation approaches can be used. These combine prior available baseline models with the ANN learning components, resulting in faster convergence and more interpretable models. The combination of the encoder-based method with model augmentation has shown potential. Thus far, however, the encoder has still been treated as a black-box function in the overall estimation process, while additional information in the form of the baseline model is available to predict the model state from past input-output data. In this paper, we propose novel encoder initialisation approaches based on the available baseline model, resulting in improved noise robustness and faster convergence compared to black-box initialisation. The performance of these initialisation methods is demonstrated on a mass-spring-damper system.
Paper Structure (14 sections, 35 equations, 3 figures, 3 tables)

This paper contains 14 sections, 35 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Identification Problem
  3. Method
  4. Model-based encoder initialisation
  5. LTI state reconstruction
  6. LTI state reconstruction with noise model
  7. Generalisation to NL baseline models
  8. Data-based encoder initialisation
  9. Simulation Example
  10. Mass Spring Damper
  11. Baseline Model
  12. Model augmentation
  13. Monte Carlo analysis
  14. Conclusion

Figures (3)

  • Figure 1: Nonlinear 2-DOF MSD with cubic damping $d_1$ and cubic spring $a_2$.
  • Figure 2: Validation loss for 5, 10 and 100 step ahead RMSE over the fitting epochs for the three encoder initialisation methods: model-based (blue), data-based (orange) and randomly initialised (green).
  • Figure 3: Distribution of simulation RMSE over the test data for 10 different model for each encoder initialisation method.