Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Policy Optimization of Mixed H2/H-infinity Control: Benign Nonconvexity and Global Optimality

Chih-Fan Pai, Yuto Watanabe, Yujie Tang, Yang Zheng

TL;DR

This paper revisits mixed H2/H-infinity control from a modern policy optimization viewpoint, including the general two-channel and single-channel cases, and builds on an Extended Convex Lifting framework that bridges nonconvex policy optimization and convex reformulations.

Abstract

Mixed H2/H-infinity control balances performance and robustness by minimizing an H2 cost bound subject to an H-infinity constraint. However, classical Riccati/LMI solutions offer limited insight into the nonconvex optimization landscape and do not readily scale to large-scale or data-driven settings. In this paper, we revisit mixed H2/H-infinity control from a modern policy optimization viewpoint, including the general two-channel and single-channel cases. One central result is that both cases enjoy a benign nonconvex structure: every stationary point is globally optimal. We characterize the H-infinity-constrained feasible set, which is open, path-connected, with boundary given exactly by policies saturating the H-infinity constraint. We also show that the mixed objective is real analytic in the interior with explicit gradient formulas. Our key analysis builds on an Extended Convex Lifting (ECL) framework that bridges nonconvex policy optimization and convex reformulations. The ECL constructions rely on non-strict Riccati inequalities that allow us to characterize global optimality. These insights reveal hidden convexity in mixed H2/H-infinity control and facilitate the design of scalable policy iteration methods in large-scale settings.

Policy Optimization of Mixed H2/H-infinity Control: Benign Nonconvexity and Global Optimality

TL;DR

This paper revisits mixed H2/H-infinity control from a modern policy optimization viewpoint, including the general two-channel and single-channel cases, and builds on an Extended Convex Lifting framework that bridges nonconvex policy optimization and convex reformulations.

Abstract

Mixed H2/H-infinity control balances performance and robustness by minimizing an H2 cost bound subject to an H-infinity constraint. However, classical Riccati/LMI solutions offer limited insight into the nonconvex optimization landscape and do not readily scale to large-scale or data-driven settings. In this paper, we revisit mixed H2/H-infinity control from a modern policy optimization viewpoint, including the general two-channel and single-channel cases. One central result is that both cases enjoy a benign nonconvex structure: every stationary point is globally optimal. We characterize the H-infinity-constrained feasible set, which is open, path-connected, with boundary given exactly by policies saturating the H-infinity constraint. We also show that the mixed objective is real analytic in the interior with explicit gradient formulas. Our key analysis builds on an Extended Convex Lifting (ECL) framework that bridges nonconvex policy optimization and convex reformulations. The ECL constructions rely on non-strict Riccati inequalities that allow us to characterize global optimality. These insights reveal hidden convexity in mixed H2/H-infinity control and facilitate the design of scalable policy iteration methods in large-scale settings.
Paper Structure (37 sections, 26 theorems, 90 equations, 2 figures, 2 tables)

This paper contains 37 sections, 26 theorems, 90 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Related work
  4. Paper outline
  5. Preliminaries
  6. System dynamics and robustness
  7. Policy optimization for mixed $\mathcal{H}_2/\mathcal{H}_\infty$ design
  8. Problem statement
  9. Basic Landscape Properties
  10. Geometry of the $\mathcal{H}_\infty$ constrained domain
  11. Analyticity of the mixed $\mathcal{H}_2/\mathcal{H}_\infty$ cost function
  12. No Spurious Stationary Points
  13. Any stationary point is globally optimal
  14. Existence of stationary points
  15. Analysis via Extended Convex Lifting
  16. ...and 22 more sections

Key Result

Lemma 1

Consider the transfer function $\mathbf{G}(s) = C(sI - A)^{-1}B$, with $A\in\mathbb{R}^{n\times n}$, $B\in\mathbb{R}^{n\times m}$, and $C\in\mathbb{R}^{p\times n}$. Then, for any $\beta>0$, the following statements are equivalent.

Figures (2)

  • Figure 1: (a) Nonconvex $\mathcal{K}_{\beta}$ with $k_{11}=k_{22}=0$ for different $\beta$ in Example \ref{['example:mix-set-nonconvexity-unbounded']}. (b)-(c) Nonconvexity and (non)coercivity of the costs in Example \ref{['example:mix-cost-nonconvex-noncoercive']}, with feasible sets $\mathcal{K}_{3.5}$ and $\mathcal{K}_{\infty}$ for (b) and (c) respectively. Red dots highlight global minima.
  • Figure 2: Optimization landscapes of $J_{\mathrm{mix}}$, $J_\infty$, and $J_{\mathrm{LQR}}$ in Example \ref{['ex:1dim-two-channel']}. (a) Infimum $J_1^\ast$ not attained (hollow red circle marks the boundary). (b) Minimum $J_2^\ast$ attained (solid red dot). (c) Single-channel case: minimum attained (solid red dot).

Theorems & Definitions (36)

  • Remark 1: Single-channel case
  • Lemma 1: Bounded real lemma zhou1996robust
  • Lemma 2
  • Example 1
  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Lemma 3
  • Example 2: Non-coercivity
  • Lemma 4
  • ...and 26 more