Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Joint Learning of Linear Time-Invariant Dynamical Systems

Aditya Modi, Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari, George Michailidis

TL;DR

This work tackles joint identification of multiple LTI systems by assuming each system’s transition matrix is a sparse linear combination of a shared basis. It introduces a bilinear joint estimator that pools data across systems, yielding finite-time error bounds that scale favorably with the number of systems $M$, basis size $k$, and trajectory length $T$, with per-system error decaying roughly as ${1}/{T}$ and sharing gains scaling as ${d^2 k}/{(M T)}$. The authors develop tight concentration results for dependent random matrices, including covariance matrices tied to the Jordan structure, and extend the analysis to robustness under misspecification. Empirically, pooling substantially improves accuracy over learning each system separately, and the theory clarifies when joint learning is advantageous, particularly in regimes with modest misspecification and well-behaved spectral properties. The results offer practical guidance for multi-system dynamics identification and lay groundwork for extending joint-learning methods to broader dynamical settings.

Abstract

Linear time-invariant systems are very popular models in system theory and applications. A fundamental problem in system identification that remains rather unaddressed in extant literature is to leverage commonalities amongst related linear systems to estimate their transition matrices more accurately. To address this problem, the current paper investigates methods for jointly estimating the transition matrices of multiple systems. It is assumed that the transition matrices are unknown linear functions of some unknown shared basis matrices. We establish finite-time estimation error rates that fully reflect the roles of trajectory lengths, dimension, and number of systems under consideration. The presented results are fairly general and show the significant gains that can be achieved by pooling data across systems in comparison to learning each system individually. Further, they are shown to be robust against model misspecifications. To obtain the results, we develop novel techniques that are of interest for addressing similar joint-learning problems. They include tightly bounding estimation errors in terms of the eigen-structures of transition matrices, establishing sharp high probability bounds for singular values of dependent random matrices, and capturing effects of misspecified transition matrices as the systems evolve over time.

Joint Learning of Linear Time-Invariant Dynamical Systems

TL;DR

This work tackles joint identification of multiple LTI systems by assuming each system’s transition matrix is a sparse linear combination of a shared basis. It introduces a bilinear joint estimator that pools data across systems, yielding finite-time error bounds that scale favorably with the number of systems , basis size , and trajectory length , with per-system error decaying roughly as and sharing gains scaling as . The authors develop tight concentration results for dependent random matrices, including covariance matrices tied to the Jordan structure, and extend the analysis to robustness under misspecification. Empirically, pooling substantially improves accuracy over learning each system separately, and the theory clarifies when joint learning is advantageous, particularly in regimes with modest misspecification and well-behaved spectral properties. The results offer practical guidance for multi-system dynamics identification and lay groundwork for extending joint-learning methods to broader dynamical settings.

Abstract

Linear time-invariant systems are very popular models in system theory and applications. A fundamental problem in system identification that remains rather unaddressed in extant literature is to leverage commonalities amongst related linear systems to estimate their transition matrices more accurately. To address this problem, the current paper investigates methods for jointly estimating the transition matrices of multiple systems. It is assumed that the transition matrices are unknown linear functions of some unknown shared basis matrices. We establish finite-time estimation error rates that fully reflect the roles of trajectory lengths, dimension, and number of systems under consideration. The presented results are fairly general and show the significant gains that can be achieved by pooling data across systems in comparison to learning each system individually. Further, they are shown to be robust against model misspecifications. To obtain the results, we develop novel techniques that are of interest for addressing similar joint-learning problems. They include tightly bounding estimation errors in terms of the eigen-structures of transition matrices, establishing sharp high probability bounds for singular values of dependent random matrices, and capturing effects of misspecified transition matrices as the systems evolve over time.
Paper Structure (19 sections, 19 theorems, 127 equations, 5 figures)

This paper contains 19 sections, 19 theorems, 127 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Joint Learning of LTI Systems
  4. Robustness to Misspecifications
  5. Numerical Illustrations
  6. Proof of thm:cov_conc
  7. Upper Bounds on Covariance Matrices
  8. Lower Bound for Covariance Matrices
  9. Proof of thm:estimation_error
  10. Proof of Estimation Error in thm:estimation_error
  11. Proof of thm:estimation_error_pert
  12. Auxiliary Probabilistic Inequalities
  13. Concluding Remarks
  14. Additional Numerical Simulations
  15. Proofs of Auxiliary Results
  16. ...and 4 more sections

Key Result

Theorem 1

Under Assumptions assum:non-explosive and assum:noise, for each system $m$, let $\underbar \Sigma_m=\underbar \lambda_m I$ and $\bar{\Sigma}_m = \bar{\lambda}_m I$, where $\underbar\lambda_{m} \coloneqq 4^{-1}\lambda_{\min}(C)T$, and Then, there is $T_0$, such that for $m \in [M]$ and $T \ge T_0$:

Figures (5)

  • Figure 1: Logarithm of the magnitude of the state vectors vs. time, for different block-sizes in the Jordan forms of the transition matrices, which is denoted by $l$ in eq:JordanDef. The exponential scaling of the state vectors with $l$ can be seen in both plots.
  • Figure 2: Per-system estimation errors vs. the number of systems $M$, for the proposed joint learning method and individual least-squares estimates of the linear dynamical systems.
  • Figure 3: Per-system estimation errors are reported vs. the number of systems $M$, for varying proportions of misspecified systems; $M^{-a}$, for $a \in \{0,0.25,0.5\}$.
  • Figure 4: Estimation and validation prediction errors versus the hyperparameter $k'$, for the true value $k=10$.
  • Figure 5: Per-system estimation errors vs. the number of systems $M$, for the proposed joint learning method and individual least-squares estimates of the linear dynamical systems.

Theorems & Definitions (35)

  • Theorem 1: Covariance matrices
  • Definition 1
  • Theorem 2
  • Theorem 3
  • Proposition 1: Bounding $\norm{x_m(t)}$
  • proof
  • Lemma 1: Upper bound on $\Sigma_m$
  • proof
  • Lemma 2: Covariance lower bound.
  • proof
  • ...and 25 more