A New Approach to Controlling Linear Dynamical Systems
Anand Brahmbhatt, Gon Buzaglo, Sofiia Druchyna, Elad Hazan
TL;DR
The paper tackles online control of linear dynamical systems under adversarial disturbances and convex Lipschitz costs. It introduces Online Spectral Control (OSC), a convex-relaxation-based method that constructs universal spectral filters from a Hankel matrix and learns a linear combination of their effects via projected Online Gradient Descent over a bounded parameter set. The main result provides a regret bound of $\text{Regret}_T(OSC,\mathcal{S}) = \frac{C_0 C_1 \sqrt{T}}{\gamma^{4}} \log^{3} \left( \frac{C_1 T d}{\gamma^{3}} \right)$ with carefully chosen memory $m$ and filter count $h$, and shows per-step runtime that scales polylogarithmically in $1/\gamma$. The analysis proves that diagonalizably stable policies can be approximated by the spectral policy class, enabling efficient learning with strong guarantees, and experiments demonstrate OSC’s superior performance over prior methods. Overall, the work offers a scalable, theoretically grounded approach to online control with adversarial disturbances and may inspire broader use of spectral relaxations in control.
Abstract
We propose a new method for controlling linear dynamical systems under adversarial disturbances and cost functions. Our algorithm achieves a running time that scales polylogarithmically with the inverse of the stability margin, improving upon prior methods with polynomial dependence maintaining the same regret guarantees. The technique, which may be of independent interest, is based on a novel convex relaxation that approximates linear control policies using spectral filters constructed from the eigenvectors of a specific Hankel matrix.