Joint Chance Constrained Optimal Control via Linear Programming

TL;DR

This paper addresses joint chance constrained optimal control over a finite horizon, seeking policies that minimize cost while ensuring a prescribed probability of satisfying safety specifications like invariance, reachability, or reach-avoid. It converts the non-Markovian constraint into a Markovian DP on an augmented state that includes binary indicators tracking safety, and then develops a two-LP approach: first, a dual LP to identify the optimal multiplier $\lambda^{\star}$, and second, a value-function LP that yields $\lambda^{\star}$-optimal policies; the optimal mixed policy is constructed by combining the cheapest and safest $\lambda^{\star}$-optimal deterministic Markov policies. Theoretical results guarantee that solving the two LPs yields an optimal joint chance constrained policy, and numerical experiments on a discretized unicycle model demonstrate exact safety on the grid and significant cost reductions compared to safe policies, with ongoing applicability to continuous spaces via basis function approximations. The work provides a scalable, provably optimal framework for finite-horizon JCCOCPs and highlights avenues for extending safety specifications and improving scalability through function approximation. $N$-step horizons, safety probability $\alpha$, and the augmented dynamics with density $\tilde{T}$ are central to the method’s formulation and guarantees.

Abstract

We establish a linear programming formulation for the solution of joint chance constrained optimal control problems over finite time horizons. The joint chance constraint may represent an invariance, reachability or reach-avoid specification that the trajectory must satisfy with a predefined probability. For finite state and action spaces, the solution is exact and our method computationally superior to approaches in the literature. For continuous state or action spaces, our linear programming formulation enables basis function approximations.

Figures (2)

• Figure 1: Augmented state dynamics for invariance (left) and reach-avoid objectives (right). Arrows indicate binary state transitions based on the annotated condition; unannotated arrows denote transitions under any condition.
• Figure 2: The plots show ten trajectories generated by the optimal mixed policies for the invariance, reachability and reach-avoid problem in Section \ref{['sec_examples']}. The black regions denote unsafe and the blue regions target states. Trajectories that achieve the objective are painted green, and red otherwise. The crossed circle marks the initial state $x_0$.

Theorems & Definitions (15)

• Lemma III.1
• proof
• Theorem III.2
• proof
• Definition III.3
• Lemma IV.1
• proof
• Theorem IV.2
• proof
• Theorem IV.3