Regret Lower Bounds for Learning Linear Quadratic Gaussian Systems

TL;DR

The paper establishes fundamental limits on learning to control unknown linear Gaussian systems with quadratic costs by deriving local minimax regret lower bounds that scale as $\sqrt{T}$. It unifies Riccati-based control with Bayesian estimation via Fisher information and Van Trees’ inequality, showing that poorer controllability or observability inflates the learning burden. The contributions include refined lower bounds for state-feedback systems, an extension to partially observed LQG settings, and an easy-to-analyze PO family with corresponding Fisher-information bounds, all while improving constants relative to prior work. The results illuminate the inseparability of control performance and identifiability, guiding algorithm design by clarifying when logarithmic regret is unattainable and how system-theoretic quantities govern learning difficulty. Practically, the work highlights how Gramians and stability properties qualitatively affect the fundamental limits of adaptive control under uncertainty.

Abstract

TWe establish regret lower bounds for adaptively controlling an unknown linear Gaussian system with quadratic costs. We combine ideas from experiment design, estimation theory and a perturbation bound of certain information matrices to derive regret lower bounds exhibiting scaling on the order of magnitude $\sqrt{T}$ in the time horizon $T$. Our bounds accurately capture the role of control-theoretic parameters and we are able to show that systems that are hard to control are also hard to learn to control; when instantiated to state feedback systems we recover the dimensional dependency of earlier work but with improved scaling with system-theoretic constants such as system costs and Gramians. Furthermore, we extend our results to a class of partially observed systems and demonstrate that systems with poor observability structure also are hard to learn to control.

Figures (1)

• Figure 1: Above, we plot the lower bound of simchowitz2020naive and compare it to ours for the scalar system in \ref{['eq:scalarsystemtobeplotted']} with $a=1$ and $b$ varying. Noise variance, $Q$ and $R$ are all chosen to be unity. We have omitted the time dependency---which is the same for both bounds---and chosen the larger of the two values appearing in the minimum in the bound of simchowitz2020naive. Our lower bound is stronger in this regime, and in fact diverges as $b\to 0$, whereas theirs tends to $0$. In other words, our bound reflects the fact that learning to control becomes harder as control authority is lost.

Theorems & Definitions (39)

• Lemma 2.1
• proof
• Remark 2.1
• Lemma 2.2
• proof
• Definition 3.1
• Definition 3.2
• Lemma 3.1
• proof
• Proposition 3.1