Strong Duality and Dual Ascent Approach to Continuous-Time Chance-Constrained Stochastic Optimal Control

TL;DR

The paper tackles continuous-time, continuous-space chance-constrained stochastic optimal control by formulating an exit-time based CC-SOC and deriving a dual problem via Lagrangian relaxation. It proves strong duality under a structural Assumption 1 and solves the dual using a path-integral dual ascent algorithm, enabling online optimization with open-loop trajectory samples. The dual function is computed through a transformed HJB PDE, with risk estimated either by a PDE-based approach or an importance-sampling scheme. Numerical experiments on 2D and 5D robotic models demonstrate the method’s effectiveness and its advantage over traditional finite-difference methods in higher dimensions, highlighting potential for real-time risk-aware planning in uncertain environments.

Abstract

The paper addresses a continuous-time continuous-space chance-constrained stochastic optimal control (SOC) problem where the probability of failure to satisfy given state constraints is explicitly bounded. We leverage the notion of exit time from continuous-time stochastic calculus to formulate a chance-constrained SOC problem. Without any conservative approximation, the chance constraint is transformed into an expectation of an indicator function which can be incorporated into the cost function by considering a dual formulation. We then express the dual function in terms of the solution to a Hamilton-Jacobi-Bellman partial differential equation parameterized by the dual variable. Under a certain assumption on the system dynamics and cost function, it is shown that a strong duality holds between the primal chance-constrained problem and its dual. The Path integral approach is utilized to numerically solve the dual problem via gradient ascent using open-loop samples of system trajectories. We present simulation studies on chance-constrained motion planning for spatial navigation of mobile robots and the solution of the path integral approach is compared with that of the finite difference method.

Figures (7)

• Figure 1: Computational domains and exit times $t_f$
• Figure 2: Robot navigation problem for the input velocity model. The start position is shown by a yellow star and the target position (the origin) by a magenta star. $100$ sample trajectories generated using optimal control policies for two values of $\Delta$ are shown. The trajectories are color-coded; blue paths collide with the obstacle or the outer boundary, while the green paths converge in the neighborhood of the magenta star.
• Figure 3: $\eta^*$ and $P_{\mathrm{fail}}(x_0,t_0, u^*(\cdot\;;{\eta^*}))$ vs $\Delta$ for input velocity model using path integral control and FDM.
• Figure 4: Input velocity model: comparison of solutions $J(x,t_0)$ and $u^*(x,t_0)$ obtained from FDM (a-d) and path integral (e-h) for $\eta=0.05$ and $\eta=0.13$. The optimal control inputs $u^*(x,t_0)$ in (b, d, f, h) are plotted together with contours of $J(x, t_0)$.
• Figure 5: Colormaps of the failure probabilities of the optimal policies synthesized for the input velocity model as functions of initial position.

Theorems & Definitions (11)

• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof