Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model order reduction of deep structured state-space models: A system-theoretic approach

Marco Forgione, Manas Mejari, Dario Piga

TL;DR

This paper introduces two regularization terms which can be incorporated into the training loss for improved model order reduction and considers modal $\ell_{1}$ and Hankel nuclear norm regularization to promote sparsity, allowing one to retain only the relevant states without sacrificing accuracy.

Abstract

With a specific emphasis on control design objectives, achieving accurate system modeling with limited complexity is crucial in parametric system identification. The recently introduced deep structured state-space models (SSM), which feature linear dynamical blocks as key constituent components, offer high predictive performance. However, the learned representations often suffer from excessively large model orders, which render them unsuitable for control design purposes. The current paper addresses this challenge by means of system-theoretic model order reduction techniques that target the linear dynamical blocks of SSMs. We introduce two regularization terms which can be incorporated into the training loss for improved model order reduction. In particular, we consider modal $\ell_1$ and Hankel nuclear norm regularization to promote sparsity, allowing one to retain only the relevant states without sacrificing accuracy. The presented regularizers lead to advantages in terms of parsimonious representations and faster inference resulting from the reduced order models. The effectiveness of the proposed methodology is demonstrated using real-world ground vibration data from an aircraft.

Model order reduction of deep structured state-space models: A system-theoretic approach

TL;DR

This paper introduces two regularization terms which can be incorporated into the training loss for improved model order reduction and considers modal and Hankel nuclear norm regularization to promote sparsity, allowing one to retain only the relevant states without sacrificing accuracy.

Abstract

With a specific emphasis on control design objectives, achieving accurate system modeling with limited complexity is crucial in parametric system identification. The recently introduced deep structured state-space models (SSM), which feature linear dynamical blocks as key constituent components, offer high predictive performance. However, the learned representations often suffer from excessively large model orders, which render them unsuitable for control design purposes. The current paper addresses this challenge by means of system-theoretic model order reduction techniques that target the linear dynamical blocks of SSMs. We introduce two regularization terms which can be incorporated into the training loss for improved model order reduction. In particular, we consider modal and Hankel nuclear norm regularization to promote sparsity, allowing one to retain only the relevant states without sacrificing accuracy. The presented regularizers lead to advantages in terms of parsimonious representations and faster inference resulting from the reduced order models. The effectiveness of the proposed methodology is demonstrated using real-world ground vibration data from an aircraft.
Paper Structure (13 sections, 16 equations, 5 figures, 2 tables)

This paper contains 13 sections, 16 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Problem Setting
  3. Deep Structured State-Space Model
  4. Model Order Reduction and Regularization
  5. Reduction by truncation and singular perturbation
  6. Regularized Linear Recurrent Units
  7. Modal $\ell_1$-regularization
  8. Hankel nuclear norm regularization
  9. Case Study
  10. Conclusions
  11. Appendix
  12. Computation of Hankel singular values
  13. Solution to a diagonal Discrete Lyapunov equation

Figures (5)

  • Figure 1: The Deep Linear Recurrent Unit architecture.
  • Figure 2: No regularization: eigenvalues magnitude (top) and Hankel singular values (bottom).
  • Figure 3: Modal $\ell_1$ regularization: eigenvalues magnitude (top) and Hankel singular values (bottom).
  • Figure 4: Hankel nuclear norm regularization: eigenvalues magnitude (top) and Hankel singular values (bottom).
  • Figure 5: Test performance obtained for increasing number of removed modes in all layers, for selected combinations of regularization and model order reduction approaches.

Theorems & Definitions (3)

  • Remark 1
  • Remark 2: $\ell_2$-regularization
  • Remark 3: $\mathcal{H}_{\infty}$-error bound