Discrete Distributionally Robust Optimal Control with Explicitly Constrained Optimization

TL;DR

The work addresses distributional uncertainty in discrete control by formulating a discrete distributionally robust optimal control (DDROC) problem with a differentiable density‑ratio ambiguity set. It introduces a one‑layer smooth convex reformulation via Lagrange multipliers $\boldsymbol{\lambda}\ge0$ and $s$, yielding a tractable surrogate that is equivalent to the original max over distributions and interpretable as a deterministic robust control problem. The approach is proven to be solvable and, under convexity, produces convex (and differentiable) objectives suitable for gradient methods; a knapsack interpretation provides intuition on the ambiguity set’s size through the parameter $c=m/(1+d)$. Numerical experiments on a patroller‑agent design demonstrate solvability and explainability, showing improvements over standard SOC in worst‑case and averaged metrics and illustrating practical applicability. The results open avenues for extending to distributional constraints and broader optimal transport balls while maintaining tractability.

Abstract

Distributionally robust optimal control (DROC) is gaining interest. This study presents a reformulation method for discrete DROC (DDROC) problems to design optimal control policies under a worst-case distributional uncertainty. The reformulation of DDROC problems impacts both the utility of tractable improvements in continuous DROC problems and the inherent discretization modeling of DROC problems. DROC is believed to have tractability issues; namely, infinite inequalities emerge over the distribution space. Therefore, investigating tractable reformulation methods for these DROC problems is crucial. One such method utilizes the strong dualities of the worst-case expectations. However, previous studies demonstrated that certain non-trivial inequalities remain after the reformulation. To enhance the tractability of DDROC, the proposed method reformulates DDROC problems into one-layer smooth convex programming with only a few trivial inequalities. The proposed method is applied to a DDROC version of a patrol-agent design problem.

Figures (1)

• Figure 1: Graph of the San Francisco data set in diaz2023distributed.

Theorems & Definitions (12)

• Remark 1: Differentiable Subset of the Optimal Transport Ball
• Theorem 2: Tractable Discrete Distributionally Robust Control Problems
• Remark 3: Solvability of the DDROC Problems
• Theorem 4: Deterministic RC Problems with Weights
• Remark 5: Size Explanation of the Ambiguity Set
• Lemma 6: Equality of the Ambiguity Set
• proof : Proof of Lemma \ref{['lem:ambiguity']}
• Lemma 7: Strong Duality of the Worst Expectation
• Remark 8: Proof Ideas of Lemma \ref{['lem:lagrangeDual']}
• proof : Proof of Lemma \ref{['lem:lagrangeDual']}