Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sub-optimality of the Separation Principle for Quadratic Control from Bilinear Observations

Yahya Sattar, Sunmook Choi, Yassir Jedra, Maryam Fazel, Sarah Dean

Abstract

We consider the problem of controlling a linear dynamical system from bilinear observations with minimal quadratic cost. Despite the similarity of this problem to standard linear quadratic Gaussian (LQG) control, we show that when the observation model is bilinear, neither does the Separation Principle hold, nor is the optimal controller affine in the estimated state. Moreover, the cost-to-go is non-convex in the control input. Hence, finding an analytical expression for the optimal feedback controller is difficult in general. Under certain settings, we show that the standard LQG controller locally maximizes the cost instead of minimizing it. Furthermore, the optimal controllers (derived analytically) are not unique and are nonlinear in the estimated state. We also introduce a notion of input-dependent observability and derive conditions under which the Kalman filter covariance remains bounded. We illustrate our theoretical results through numerical experiments in multiple synthetic settings.

Sub-optimality of the Separation Principle for Quadratic Control from Bilinear Observations

Abstract

We consider the problem of controlling a linear dynamical system from bilinear observations with minimal quadratic cost. Despite the similarity of this problem to standard linear quadratic Gaussian (LQG) control, we show that when the observation model is bilinear, neither does the Separation Principle hold, nor is the optimal controller affine in the estimated state. Moreover, the cost-to-go is non-convex in the control input. Hence, finding an analytical expression for the optimal feedback controller is difficult in general. Under certain settings, we show that the standard LQG controller locally maximizes the cost instead of minimizing it. Furthermore, the optimal controllers (derived analytically) are not unique and are nonlinear in the estimated state. We also introduce a notion of input-dependent observability and derive conditions under which the Kalman filter covariance remains bounded. We illustrate our theoretical results through numerical experiments in multiple synthetic settings.

Paper Structure

This paper contains 22 sections, 5 theorems, 40 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Setting
  3. Bilinear Observation Optimal Control Problem
  4. Input-Dependent State Estimation
  5. Main Results
  6. Sub-optimality of the Separation Principle
  7. Sufficient Conditions for Uniformly Bounded Cost
  8. Proofs of Main Results
  9. Proof of Theorem \ref{['thrm:negative result']}
  10. Proof of Theorem \ref{['thrm:nonlinear special case']}
  11. Proof of Proposition \ref{['prop:perpendicular']}
  12. Derivation of Equation \ref{['eqn:u_Tm2_Bellman']}
  13. Numerical Experiments
  14. Experimental Setups
  15. Scalar Example
  16. ...and 7 more sections

Key Result

Lemma 1

Under Assumption assump noise, initial, cost(i),(ii), the Kalman filtering algorithm gives:

Figures (3)

  • Figure 1: The landscape of $f_0(u) = f_{LQG}(u) + g(u)$ depends on the distance between the global minima of $f_{LQG}(u):=\alpha u^2 + 2 \beta \hat{x}_0 u$, and the global maxima of $g(u):=\frac{\gamma}{(C_0+C_1 u)^2 + \kappa}$, where $\alpha, \beta, \gamma, \kappa$ are as defined in \ref{['eqn:alpha beta gamma kappa']}. When this distance, captured by $|C_0/C_1 - \beta \hat{x}_0/\alpha|$ is large $u_0^\star$ and $u_0^{\rm LQG}$ give similar performance. On the other hand, when $|C_0/C_1 - \beta \hat{x}_0/\alpha|$ is small, $u_0^\star$ and $u_0^{\rm LQG}$ give very different performance.
  • Figure 2: Double Integrator. Bilinear observations $(C_1=1)$ incur more cost (a) than LQG due to the negative effects of small input (b) on state estimation (c,d).
  • Figure 3: For fixed ${\boldsymbol{A}}$, ${\boldsymbol{B}}$, ${\boldsymbol{C}}_1,\dots,{\boldsymbol{C}}_p$, two plots correspond to different ${\boldsymbol{C}}_0$.

Theorems & Definitions (12)

  • Definition 1: Admissible Policy
  • Lemma 1: Optimality of KF
  • Definition 2: Separation principle
  • Theorem 1: Counter example
  • Theorem 2: Nonlinear controller
  • Definition 3: Complete observability
  • Lemma 2: Bounded error covariance
  • Proposition 1: Sufficient condition
  • proof
  • proof
  • ...and 2 more