Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BLISS: Global Blind Identification of Linear Systems with Sparse Inputs

Kyle Poe, Uday Kiran Reddy Tadipatri, Benjamin D. Haeffele, Rene Vidal

Abstract

Linear system identification and sparse dictionary learning can both be seen as structured matrix factorization problems. However, these two problems have historically been studied in isolation by the systems theory and machine learning communities. Although linear system identification enjoys a mature theory when inputs are known, blind linear system identification remains poorly understood beyond restrictive settings. In contrast, complete sparse dictionary learning has recently benefited from strong global identifiability results and scalable nonconvex algorithms. In this work, we bridge these two areas by showing that under a sparse input assumption, fully observed blind system identification becomes a generalization of complete dictionary learning. This connection allows us to develop global identifiability guarantees for blind system identification, by leveraging techniques from the complete dictionary learning literature. We further show empirically that a principled application of the alternating direction method of multipliers can globally recover the ground-truth system from a single trajectory, provided sufficient samples and input sparsity.

BLISS: Global Blind Identification of Linear Systems with Sparse Inputs

Abstract

Linear system identification and sparse dictionary learning can both be seen as structured matrix factorization problems. However, these two problems have historically been studied in isolation by the systems theory and machine learning communities. Although linear system identification enjoys a mature theory when inputs are known, blind linear system identification remains poorly understood beyond restrictive settings. In contrast, complete sparse dictionary learning has recently benefited from strong global identifiability results and scalable nonconvex algorithms. In this work, we bridge these two areas by showing that under a sparse input assumption, fully observed blind system identification becomes a generalization of complete dictionary learning. This connection allows us to develop global identifiability guarantees for blind system identification, by leveraging techniques from the complete dictionary learning literature. We further show empirically that a principled application of the alternating direction method of multipliers can globally recover the ground-truth system from a single trajectory, provided sufficient samples and input sparsity.

Paper Structure

This paper contains 14 sections, 9 theorems, 41 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Notation
  3. Background
  4. Fully Observed Linear System Identification
  5. Identifiability of Complete Dictionary Learning
  6. Blind Identification by Volume Minimization
  7. BLISS: Algorithmic Implementation
  8. Experiments
  9. Data generation
  10. Algorithm
  11. Settings
  12. Discussion
  13. Conclusions
  14. Proofs and Lemmata

Key Result

Corollary B.1

Suppose the system $(\bm{A}_\natural, \bm{B}_\natural)$ is controllable, and that $\bm{u}$ is persistently exciting of order $n+1$: Then $\text{rank}\,[\bm{X}, \bm{U}] = n + m$. $\blacktriangleleft$$\blacktriangleleft$

Figures (4)

  • Figure 3: A visualization of the persistent scattering condition, viewing $\mathbb{R}^n \times \mathbb{R}^m \to \mathbb{R} \times \mathbb{R}^2$, with the first (states) coordinate vertical, and remaining coordinates (inputs) in the $x, y$ plane. The set $\mathcal{U}$, inscribed in $\{0\} \times [-1, 1]^2$, is sufficiently scattered relative to the $x, y$ plane. The set $\mathcal{Z}$ (the polyhedron outlined in solid blue) is not sufficiently scattered, but contains an ellipse (in red) such that the intersection of its boundary with the ellipse is $\{(0, \pm1, 0), (0, 0, \pm 1)\}$.
  • Figure 4: Probability of successful recovery ($P(\text{success})$) as a function of sparsity fraction $s/m$ and trajectory length $T$ for a stable, and controllable system with $n=100$, $m=25$, and 50 trials per cell.
  • Figure 5: BLISS algorithm convergence for different initializations of the penalty parameter $\rho_0$. Solid lines correspond to oracle-scaled initializations $\rho^{(0)} = c\rho_{\text{oracle}}$ for $c \in \{1, 2, 5, 10\}$, where $\rho_{\text{oracle}} = \sigma_{\max}(( \bm{X}_+\bm{P}_{\bm{X}}^\perp)^\dagger \bm{B}_\natural)^2$, while dashed lines show fixed initializations $\rho^{(0)} \in \{1, 2\}$. Shaded regions indicate $\pm 1$ standard deviation over $50$ independent trials. The black dashed horizontal line indicates residual level of 1e-6.
  • Figure 6: Probability of successful recovery vs. input sparsity $s/m$ for the stable, controllable case ($n=100$, $m=25$, $T=1000$, 50 trials per point). Solid lines: unknown $A$ or blind system identification; dashed lines: known $A$ (oracle) or dictionary learning.

Theorems & Definitions (25)

  • Corollary B.1: Willems willems-et-al-scl05
  • Definition B.1: SP-Equivalent
  • Definition B.2: Sufficient Scattering
  • Theorem B.1: Hu and Huang hu2023global
  • Remark C.1
  • proof
  • Definition C.1: Fundamental Zonotope
  • Definition C.2: Persistently Scattering
  • Proposition C.1: Support Function Characterization
  • proof
  • ...and 15 more