Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Avoiding Semi-Infinite Programming in Distributionally Robust Control Based on Mean-Variance Metrics

Yuma Shida, Yuji Ito

TL;DR

This reformulation reduces the original DRC problem to a discounted mean-variance cost optimization problem, and establishes upper bounds, and demonstrates the equivalence between distributionally robust optimization problems and mean-variance minimization problems.

Abstract

Conventional stochastic control methods have several limitations. They focus on optimizing the average performance and, in some cases, performance variability; however, their problem settings still require an explicit specification of the probability distributions that determine the system's stochastic behavior. Distributionally robust control (DRC) methods have recently been developed to address these challenges. However, many DRC approaches involve handling infinitely many inequalities. For instance, DRC problems based on the Wasserstein distance are commonly obtained by solving semi-infinite programming (SIP) problems. Our proposed method eliminates the need for SIP when solving discrete-time, discounted, distributionally robust optimal control problems. By introducing a penalty term based on a specific distributional distance, we establish upper bounds, and under appropriate conditions, demonstrate the equivalence between distributionally robust optimization problems and mean-variance minimization problems. This reformulation reduces the original DRC problem to a discounted mean-variance cost optimization problem. In linear-quadratic regulator settings, the corresponding control laws are obtained by solving the Riccati equation. Numerical experiments demonstrate that the theoretical maximum value of the discounted cumulative cost for the proposed method is lower than that for the conventional method.

Avoiding Semi-Infinite Programming in Distributionally Robust Control Based on Mean-Variance Metrics

TL;DR

This reformulation reduces the original DRC problem to a discounted mean-variance cost optimization problem, and establishes upper bounds, and demonstrates the equivalence between distributionally robust optimization problems and mean-variance minimization problems.

Abstract

Conventional stochastic control methods have several limitations. They focus on optimizing the average performance and, in some cases, performance variability; however, their problem settings still require an explicit specification of the probability distributions that determine the system's stochastic behavior. Distributionally robust control (DRC) methods have recently been developed to address these challenges. However, many DRC approaches involve handling infinitely many inequalities. For instance, DRC problems based on the Wasserstein distance are commonly obtained by solving semi-infinite programming (SIP) problems. Our proposed method eliminates the need for SIP when solving discrete-time, discounted, distributionally robust optimal control problems. By introducing a penalty term based on a specific distributional distance, we establish upper bounds, and under appropriate conditions, demonstrate the equivalence between distributionally robust optimization problems and mean-variance minimization problems. This reformulation reduces the original DRC problem to a discounted mean-variance cost optimization problem. In linear-quadratic regulator settings, the corresponding control laws are obtained by solving the Riccati equation. Numerical experiments demonstrate that the theoretical maximum value of the discounted cumulative cost for the proposed method is lower than that for the conventional method.
Paper Structure (11 sections, 5 theorems, 30 equations, 1 figure)

This paper contains 11 sections, 5 theorems, 30 equations, 1 figure.

Table of Contents

  1. INTRODUCTION
  2. Problem Settings
  3. DRO Problem
  4. DRC Problems
  5. Proposed Method
  6. Computing Solutions to DRO problem
  7. Computing Solutions to DRC
  8. Computing Solutions via Riccati Equations
  9. Proofs of Theorems
  10. Numerical Experiments
  11. CONCLUSIONS

Key Result

Proposition 1

Suppose that there exists a value function $V_\mathrm{DRC}(\boldsymbol{x})$ that satisfies the Bellman equation (eq:DRCBellman), that $\boldsymbol{u}^*(\boldsymbol{x})$ is a minimizer to (eq:DRCBellman) for every $\boldsymbol{x}\in\mathcal{X}$, and that the discount factor $\alpha$ is sufficiently s

Figures (1)

  • Figure 1: Numerical results for the inverted pendulum on a cart. The circles and crosses represent the results of the proposed and conventional methods, respectively. The vertical axis represents the theoretical maximum of the discounted cumulative cost as shown in Corollary \ref{['cor:equivalence']} (ii), and the horizontal axis represents the coefficient for the distributional distance penalty in the distributionally robust control problem.

Theorems & Definitions (18)

  • Proposition 1: Bellman Equations of DRC Problems
  • proof : Proof of Proposition \ref{['prop:DRC']}
  • Theorem 2: Reformulation of DRO Problem
  • Remark 3: Proof of Theorem \ref{['thm:DRO']}
  • Remark 4: Equivalence Between DRO and Mean--Variance Minimization Problems
  • Remark 5: Mean--Variance Function
  • Proposition 6: Bellman Equations of Mean--Variance-Type Control Problems
  • proof : Proof of Proposition \ref{['prop:MVC']}
  • Corollary 7: Reformulations of DRC Problems
  • proof : Proof of Corollary \ref{['cor:equivalence']}
  • ...and 8 more