Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sample Complexity Bounds for Linear System Identification from a Finite Set

Nicolas Chatzikiriakos, Andrea Iannelli

TL;DR

This work addresses finite-sample identification of a true LTI system from a finite model set using trajectory data. It develops non-asymptotic, instance-specific upper bounds for the MLE's sample complexity and a information-theoretic lower bound that hold without stability assumptions, supported by both analytical and numerical results. The analysis leverages a block martingale small-ball condition to manage correlated data and demonstrates how excitation, noise, and input directions shape sample efficiency. The numerical example corroborates the theoretical insights, highlighting the importance of excitation direction and showing MLE's practical advantages over OLS. Overall, the paper establishes fundamental limits and design principles for finite-hypothesis linear system identification.

Abstract

This paper considers a finite sample perspective on the problem of identifying an LTI system from a finite set of possible systems using trajectory data. To this end, we use the maximum likelihood estimator to identify the true system and provide an upper bound for its sample complexity. Crucially, the derived bound does not rely on a potentially restrictive stability assumption. Additionally, we leverage tools from information theory to provide a lower bound to the sample complexity that holds independently of the used estimator. The derived sample complexity bounds are analyzed analytically and numerically.

Sample Complexity Bounds for Linear System Identification from a Finite Set

TL;DR

This work addresses finite-sample identification of a true LTI system from a finite model set using trajectory data. It develops non-asymptotic, instance-specific upper bounds for the MLE's sample complexity and a information-theoretic lower bound that hold without stability assumptions, supported by both analytical and numerical results. The analysis leverages a block martingale small-ball condition to manage correlated data and demonstrates how excitation, noise, and input directions shape sample efficiency. The numerical example corroborates the theoretical insights, highlighting the importance of excitation direction and showing MLE's practical advantages over OLS. Overall, the paper establishes fundamental limits and design principles for finite-hypothesis linear system identification.

Abstract

This paper considers a finite sample perspective on the problem of identifying an LTI system from a finite set of possible systems using trajectory data. To this end, we use the maximum likelihood estimator to identify the true system and provide an upper bound for its sample complexity. Crucially, the derived bound does not rely on a potentially restrictive stability assumption. Additionally, we leverage tools from information theory to provide a lower bound to the sample complexity that holds independently of the used estimator. The derived sample complexity bounds are analyzed analytically and numerically.
Paper Structure (19 sections, 4 theorems, 45 equations, 1 table)

This paper contains 19 sections, 4 theorems, 45 equations, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem setup
  4. Maximum likelihood estimation
  5. Finite sample identification
  6. A sample complexity upper bound
  7. Lower bounding the empirical risk of $\theta \neq \theta_*$
  8. Upper bounding the empirical risk of $\theta_0$
  9. Leveraging concentration and anti-concentration
  10. A sample complexity lower bound
  11. Analysis of the sample complexity bounds
  12. Excitation and noise level
  13. Excitation directions
  14. Sample efficiency
  15. Numerical example
  16. ...and 4 more sections

Key Result

Proposition 1

Let Assumption ass:Gaussian hold, let $z_t^i$ be defined according to eq:defz and define Then the $\{\mathcal{F}_t\}_{t=1}^T$-adapted random process $(z_t^i)_{t=1}^T$ satisfies the $\left( k, \Sigma_{z_{k/2}}^i, 3/20\right)$-BMSB condition for all ${k\in [1,T]}$.

Theorems & Definitions (9)

  • Definition 1: Block Martingale Small-Ball simchowitz2018learning
  • Proposition 1
  • Theorem 1
  • proof
  • Remark 1
  • Definition 2: $\delta$-stable algorithms
  • Theorem 2
  • Corollary 1
  • proof