Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems

Florian Messerer, Katrin Baumgärtner, Armin Nurkanović, Moritz Diehl

Abstract

We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.

Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems

Abstract

We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.
Paper Structure (18 sections, 6 theorems, 54 equations, 11 figures)

This paper contains 18 sections, 6 theorems, 54 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Contribution and outline
  3. Notation
  4. Ordinary differential equations with discontinuous right-hand side
  5. Nondegenerate switching cases
  6. Numerical integration
  7. Solution sensitivities
  8. Initial value problems with uncertain initial value
  9. Linearization-based uncertainty propagation for smooth dynamics
  10. Linearization-based uncertainty propagation for nonsmooth dynamics
  11. Normalization-based uncertainty propagation
  12. Uncertainty propagation for scalar piecewise constant systems
  13. Case study: the switched normal distribution
  14. Approximate uncertainty dynamics via normalization
  15. Uncertainty propagation for piecewise affine systems
  16. ...and 3 more sections

Key Result

lemma 1

Consider a distribution in state space, $x\sim\mathcal{X}$, that evolves according to $\dot x = f(x)$. Consider also a normal distribution on the same space, $\mathcal{N}(\mu,\Sigma)$. Then, the time derivative of mean and covariance can be written as with the error terms $\delta_f$, $\Delta_f$ defined in eq:error_def.

Figures (11)

  • Figure 1: Crossing the discontinuity. Left: The state trajectories from Example \ref{['ex:crossing1D_nom']}. The switch at $x=0$ leads to a kink in each trajectory and to a scaling of their distances with respect to each other. Right: The corresponding integrator map $x(T; x_0)$ as a function of $x_0$, which is piecewise affine. For $-6\leq x_0 \leq 0$, the discontinuity is crossed within the integration interval. The blue lines visualize the corresponding mapping of the initial states from the left-hand side plot. The scaling of the distances is a consequence of the slope of the integrator map in the corresponding region.
  • Figure 2: Sliding mode. Left: The state trajectories from Example \ref{['ex:sliding1D_nom']}. Once a trajectory reaches $x=0$ it stays there, which leads to narrowing of the distances over time. Right: The corresponding integrator map $x(T; x_0)$ as a function of $x_0$, which is piecewise affine. The flat region in the center corresponds to the values of $x_0$ such that $x(T, x_0)$ reaches the switching surface. The blue lines visualize the corresponding mapping of the initial states from the left-hand side plot.
  • Figure 3: Left: The means of the two imagined normal distributions $\mathcal{N}(\mu_i(t), \sigma_i^2)$, $i=1,2$, compared to the mean of the exactly propagated distribution. The shaded regions indicate $3\sigma$ on each side of the mean. The dotted lines indicate the 99.7% probability mass corresponding to the original $\pm3\sigma$ region. Right: The cumulative density function $\bar{\Phi}_\mathrm{s}(x; \mu_1(t), \mu_2(t), \sigma_1, \sigma_2)$ of the switched normal distribution for various time points as the distribution crosses the switch at $x=0$. The arrows indicate the state dynamics.
  • Figure 4: Left: The means of the two imagined normal distributions $\mathcal{N}(\mu_i(t), \sigma_i^2)$, $i=1,2$, compared to the mean of the exactly propagated distribution. The shaded regions indicate $3\sigma$ on each side of the mean. The dotted lines indicate the 99.7% probability mass corresponding to the original $\pm3\sigma$ region. Right: The cumulative density function $\bar{\Phi}^\prime_\mathrm{s}(x; \mu_1(t), \mu_2(t), \sigma_1, \sigma_2)$ of the modified switched normal distribution for various time points as the distribution crosses the switch at $x=0$. On both sides of $x=0$, the probability mass is transported towards the origin, where it accumulates. The arrows indicate the state dynamics.
  • Figure 5: The approximated mean and variance dynamics \ref{['eq:dyn_approx_1Dconst']} for the system from Example \ref{['ex:crossing']}.
  • ...and 6 more figures

Theorems & Definitions (11)

  • lemma 1
  • proof
  • lemma 2
  • proof
  • lemma 3
  • proof
  • proposition 1
  • proof
  • proposition 2
  • corollary 1
  • ...and 1 more