Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimality of Linear Policies for Distributionally Robust Linear Quadratic Gaussian Regulator with Stationary Distributions

Nicolas Lanzetti, Antonio Terpin, Florian Dörfler

TL;DR

The paper addresses distributionally robust LQG control for discrete-time, time-varying systems with noisy measurements under Wasserstein ambiguity around Gaussian references. It proves that, when reference noises are Gaussian, output-feedback linear policies are optimal and a Nash equilibrium exists between a controller and an adversary, with worst-case noises being affine pushes of the reference laws; a quasi-closed-form solution for worst-case distributions is also derived for non-Gaussian references. The analysis relies on a purified-output reformulation and first-order optimality in probability space to characterize worst-case pushforward noises and to establish the sufficiency of linear policies in the Gaussian case. The results yield a less conservative DR-LQG framework and provide practical computational pathways (iterated best response or gradient-based min–max methods) for computing optimal policies under distributional uncertainty.

Abstract

We prove that output-feedback linear policies remain optimal for solving the Linear Quadratic Gaussian regulation problem in the face of worst-case process and measurement noise distributions when these are independent, stationary, and known to be within a radius (in the Wasserstein sense) to some reference zero-mean Gaussian noise distributions. Additionally, we establish the existence of a Nash equilibrium of the zero-sum game between a control engineer, who minimizes control cost, and a fictitious adversary, who chooses the noise distributions that maximize this cost. For general (possibly non-Gaussian) reference noise distributions, we establish a quasi closed-form solution for the worst-case distributions against linear policies. Our work provides a less conservative alternative compared to recent work in distributionally robust control.

Optimality of Linear Policies for Distributionally Robust Linear Quadratic Gaussian Regulator with Stationary Distributions

TL;DR

The paper addresses distributionally robust LQG control for discrete-time, time-varying systems with noisy measurements under Wasserstein ambiguity around Gaussian references. It proves that, when reference noises are Gaussian, output-feedback linear policies are optimal and a Nash equilibrium exists between a controller and an adversary, with worst-case noises being affine pushes of the reference laws; a quasi-closed-form solution for worst-case distributions is also derived for non-Gaussian references. The analysis relies on a purified-output reformulation and first-order optimality in probability space to characterize worst-case pushforward noises and to establish the sufficiency of linear policies in the Gaussian case. The results yield a less conservative DR-LQG framework and provide practical computational pathways (iterated best response or gradient-based min–max methods) for computing optimal policies under distributional uncertainty.

Abstract

We prove that output-feedback linear policies remain optimal for solving the Linear Quadratic Gaussian regulation problem in the face of worst-case process and measurement noise distributions when these are independent, stationary, and known to be within a radius (in the Wasserstein sense) to some reference zero-mean Gaussian noise distributions. Additionally, we establish the existence of a Nash equilibrium of the zero-sum game between a control engineer, who minimizes control cost, and a fictitious adversary, who chooses the noise distributions that maximize this cost. For general (possibly non-Gaussian) reference noise distributions, we establish a quasi closed-form solution for the worst-case distributions against linear policies. Our work provides a less conservative alternative compared to recent work in distributionally robust control.

Paper Structure

This paper contains 12 sections, 2 theorems, 54 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Motivating example
  3. Challenges
  4. Main result
  5. Proofs
  6. Proof of \ref{['lemma:optimality-conditions']}
  7. Proof of \ref{['thm:optimal-solution']}
  8. A priori upper bound with linear feedback policies
  9. A priori lower bound with Gaussian distributions
  10. Nash equilibrium and sufficiency of linear policies
  11. Acknowledgements
  12. Convenient matrices definitions

Key Result

Theorem 1

Let $\hat{\mu}_{}$ and $\hat{\nu}_{}$ be Gaussian the reference process and noise distributions, with zero means and covariances $0 \prec \hat{V}_{} \in \mathbb R^{n\times n}, 0 \prec \hat{W}_{} \in \mathbb R^{p\times p}$. Let also $r_{v_{}}, r_{w_{}} > 0$. Then, (i) it suffices to optimize over lin In particular, let $\boldsymbol{\pi}_{\mathrm{lin}}^\ast$ be the optimal linear policy in eq:minmax

Figures (1)

  • Figure 1: Our distributionally robust acr:lqg control problem can be interpreted as a Stackelberg game between a control engineer, trying to minimize the control cost, and an adversarial nature, seeking to maximize this control cost. The game is sequential: The control engineer chooses first their feedback policy; nature then selects the distributions of process and measurement noise. Under mild assumptions, we show that (i) it is optimal for the control engineer to select a linear feedback policy, (ii) nature reacts with Gaussian adversarial probability distributions and (iii) the game has a Nash equilibrium.

Theorems & Definitions (2)

  • Theorem 1: Characterization of the optimal policy
  • Lemma 1: Characterization of the worst-case distributions