Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Stabilize Unknown LTI Systems on a Single Trajectory under Stochastic Noise

Ziyi Zhang, Yorie Nakahira, Guannan Qu

TL;DR

This work tackles stabilizing an unknown noisy LTI system from a single trajectory, where instability concentrates in a low-dimensional subspace. It introduces LTS0-N, a model-based method that first identifies the unstable subspace via SVD, then estimates the unstable dynamics and the effect of control on that subspace, and finally designs a τ-hop stabilizing controller. Theoretical results show stabilization with a state-norm upper bound of $2^{O(k \log k + \log(n-k) + m - \log\text{gap})}$, avoiding the traditional exponential blow-up in the full state dimension $n$, especially when $m=O(k)$. The approach is robust to stochastic noise and under-actuated settings, and simulations demonstrate faster stabilization than existing benchmarks on a single trajectory, highlighting practical relevance for high-dimensional, uncertain systems.

Abstract

We study the problem of learning to stabilize unknown noisy Linear Time-Invariant (LTI) systems on a single trajectory. It is well known in the literature that the learn-to-stabilize problem suffers from exponential blow-up in which the state norm blows up in the order of $Θ(2^n)$ where $n$ is the state space dimension. This blow-up is due to the open-loop instability when exploring the $n$-dimensional state space. To address this issue, we develop a novel algorithm that decouples the unstable subspace of the LTI system from the stable subspace, based on which the algorithm only explores and stabilizes the unstable subspace, the dimension of which can be much smaller than $n$. With a new singular-value-decomposition(SVD)-based analytical framework, we prove that the system is stabilized before the state norm reaches $2^{O(k \log n)}$, where $k$ is the dimension of the unstable subspace. Critically, this bound avoids exponential blow-up in state dimension in the order of $Θ(2^n)$ as in the previous works, and to the best of our knowledge, this is the first paper to avoid exponential blow-up in dimension for stabilizing LTI systems with noise.

Learning to Stabilize Unknown LTI Systems on a Single Trajectory under Stochastic Noise

TL;DR

This work tackles stabilizing an unknown noisy LTI system from a single trajectory, where instability concentrates in a low-dimensional subspace. It introduces LTS0-N, a model-based method that first identifies the unstable subspace via SVD, then estimates the unstable dynamics and the effect of control on that subspace, and finally designs a τ-hop stabilizing controller. Theoretical results show stabilization with a state-norm upper bound of , avoiding the traditional exponential blow-up in the full state dimension , especially when . The approach is robust to stochastic noise and under-actuated settings, and simulations demonstrate faster stabilization than existing benchmarks on a single trajectory, highlighting practical relevance for high-dimensional, uncertain systems.

Abstract

We study the problem of learning to stabilize unknown noisy Linear Time-Invariant (LTI) systems on a single trajectory. It is well known in the literature that the learn-to-stabilize problem suffers from exponential blow-up in which the state norm blows up in the order of where is the state space dimension. This blow-up is due to the open-loop instability when exploring the -dimensional state space. To address this issue, we develop a novel algorithm that decouples the unstable subspace of the LTI system from the stable subspace, based on which the algorithm only explores and stabilizes the unstable subspace, the dimension of which can be much smaller than . With a new singular-value-decomposition(SVD)-based analytical framework, we prove that the system is stabilized before the state norm reaches , where is the dimension of the unstable subspace. Critically, this bound avoids exponential blow-up in state dimension in the order of as in the previous works, and to the best of our knowledge, this is the first paper to avoid exponential blow-up in dimension for stabilizing LTI systems with noise.
Paper Structure (18 sections, 22 theorems, 158 equations, 1 figure, 1 table, 1 algorithm)

This paper contains 18 sections, 22 theorems, 158 equations, 1 figure, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Preliminaries
  4. Decomposition of the State Space
  5. $\tau$-hop Control
  6. Main Results
  7. Algorithm
  8. Stability Guarantee
  9. Proof Outline
  10. Numerical simulation
  11. Proof of Theorem \ref{['thm:projection']}
  12. Auxillary Lemmas for Step 1
  13. Proof of Auxiliary Lemmas for Appendix \ref{['Appendix:D1']}
  14. Solution to the Least Square Problem in Stage 2
  15. Bounding $\left\lVert\hat{B}_\tau - B_\tau\right\rVert$
  16. ...and 3 more sections

Key Result

Theorem 4.2

Given a noisy LTI system $x_{t+1} = Ax_t + Bu_t + \eta_t$ subject to assumption:eigengap, assumption:pdf, and additionally, $|\lambda_1| |\lambda_{k+1}| < 1$. Further, denote $\text{gap} := \left|\prod_{\substack{m_1 \neq m_2,\\m_1, m_2 \in \{1,\dots,k\} }}(\lambda_{m_1}^{-1} - \lambda_{m_2}^{-1})\r before termination. Here the big-O notation only shows dependence on $k,m$ and $n$, while omitting

Figures (1)

  • Figure 1: In (a), the line shows the average steps it takes to stabilize the system, and the shadow area shows the standard deviation. In (b), the trajectory of our algorithm, the algorithm in LTI, the black-box controller in Chen07 and a self-turning regulator in Astrom96 are compared in a randomly generated LTI system with $n = 128$, $k = 4$, $m = 3$, and $\sigma = 0.01$.

Theorems & Definitions (40)

  • Definition 2.1: Stabilizing controller
  • Remark 4.1
  • Theorem 4.2
  • Theorem 5.1
  • Proposition 5.2
  • Lemma 5.3
  • proof
  • Lemma A.1
  • Lemma B.1
  • proof : proof of \ref{['lemm:D1_bound_final']}
  • ...and 30 more