Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stable Linear Subspace Identification: A Machine Learning Approach

Loris Di Natale, Muhammad Zakwan, Bratislav Svetozarevic, Philipp Heer, Giancarlo Ferrari-Trecate, Colin N. Jones

TL;DR

This work addresses stable linear system identification by integrating ML tooling, notably automatic differentiation, with a new LMI-based free parametrization of Schur matrices to guarantee stability. It introduces SIMBa, a backpropagation-driven framework that minimizes multi-step-ahead prediction error while enforcing $A$ to be Schur-stable, enabling robust MIMO SI with missing data and GPU acceleration. Empirically, SIMBa achieves state-of-the-art performance and stability across both simulated and real-world datasets, often improving over traditional stable SI methods by 25% or more at the cost of higher computation. The authors provide open-source software and outline extensions to structured nonlinear models and Koopman-based approaches, highlighting practical impact for reliable and scalable linear system identification.

Abstract

Machine Learning (ML) and linear System Identification (SI) have been historically developed independently. In this paper, we leverage well-established ML tools - especially the automatic differentiation framework - to introduce SIMBa, a family of discrete linear multi-step-ahead state-space SI methods using backpropagation. SIMBa relies on a novel Linear-Matrix-Inequality-based free parametrization of Schur matrices to ensure the stability of the identified model. We show how SIMBa generally outperforms traditional linear state-space SI methods, and sometimes significantly, although at the price of a higher computational burden. This performance gap is particularly remarkable compared to other SI methods with stability guarantees, where the gain is frequently above 25% in our investigations, hinting at SIMBa's ability to simultaneously achieve state-of-the-art fitting performance and enforce stability. Interestingly, these observations hold for a wide variety of input-output systems and on both simulated and real-world data, showcasing the flexibility of the proposed approach. We postulate that this new SI paradigm presents a great extension potential to identify structured nonlinear models from data, and we hence open-source SIMBa on https://github.com/Cemempamoi/simba.

Stable Linear Subspace Identification: A Machine Learning Approach

TL;DR

This work addresses stable linear system identification by integrating ML tooling, notably automatic differentiation, with a new LMI-based free parametrization of Schur matrices to guarantee stability. It introduces SIMBa, a backpropagation-driven framework that minimizes multi-step-ahead prediction error while enforcing to be Schur-stable, enabling robust MIMO SI with missing data and GPU acceleration. Empirically, SIMBa achieves state-of-the-art performance and stability across both simulated and real-world datasets, often improving over traditional stable SI methods by 25% or more at the cost of higher computation. The authors provide open-source software and outline extensions to structured nonlinear models and Koopman-based approaches, highlighting practical impact for reliable and scalable linear system identification.

Abstract

Machine Learning (ML) and linear System Identification (SI) have been historically developed independently. In this paper, we leverage well-established ML tools - especially the automatic differentiation framework - to introduce SIMBa, a family of discrete linear multi-step-ahead state-space SI methods using backpropagation. SIMBa relies on a novel Linear-Matrix-Inequality-based free parametrization of Schur matrices to ensure the stability of the identified model. We show how SIMBa generally outperforms traditional linear state-space SI methods, and sometimes significantly, although at the price of a higher computational burden. This performance gap is particularly remarkable compared to other SI methods with stability guarantees, where the gain is frequently above 25% in our investigations, hinting at SIMBa's ability to simultaneously achieve state-of-the-art fitting performance and enforce stability. Interestingly, these observations hold for a wide variety of input-output systems and on both simulated and real-world data, showcasing the flexibility of the proposed approach. We postulate that this new SI paradigm presents a great extension potential to identify structured nonlinear models from data, and we hence open-source SIMBa on https://github.com/Cemempamoi/simba.
Paper Structure (14 sections, 1 theorem, 14 equations, 2 figures, 1 table)

This paper contains 14 sections, 1 theorem, 14 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Subspace identification for linear systems
  3. Enforcing stability
  4. Contribution
  5. Preliminaries
  6. Free Parametrization of Schur Matrices
  7. A Short Introduction to SIMBa
  8. Optimization problem
  9. Initialization
  10. Numerical Experiments
  11. Comparison using random stable models
  12. Performance on real-world input-output data
  13. A note on the training time
  14. Conclusion

Key Result

Proposition 1

For any $W\in\mathbb{R}^{2n \times 2n}$, $V\in\mathbb{R}^{n \times n}$, $\tilde{\epsilon}\in\mathbb{R}$, and given $0< \gamma \leq 1$, let for $\epsilon = \exp{(\tilde{\epsilon})}$. Then is Schur stable with $|\lambda_i(A)|<\gamma, \forall i=1,...,n$.

Figures (2)

  • Figure 1: Performance of input-output state-space SI methods on $50$ randomly generated systems, where the MSEs have been normalized by the best-obtained error for each system. The performance of SIMBa (ours) is plotted in green, other stable SI methods in blue, while red indicates methods without stability guarantees. Key metrics are reported in Table \ref{['tab:random']} for clarity.
  • Figure 2: MSE of the different methods on the power plant test data for different choices of state dimension $n$, normalized by the best performance obtained by MATLAB's SI toolbox (red crosses and triangles). Black and green data show the performance of SIMBa over $10$ runs with random initialization for shorter (SIMBa) and longer (SIMBa_L) training times, respectively. Finally, blue crosses triangles represent the performance of one initialized version of SIMBa, SIMBa_i, and a prolonged version SIMBa_iL.

Theorems & Definitions (3)

  • Proposition 1
  • proof
  • Remark 1