Stochastic Reachability of Uncontrolled Systems via Probability Measures: Approximation via Deep Neural Networks

TL;DR

This work recasts stochastic reachability for autonomous, uncontrolled systems into a measure-theoretic problem, characterizing reachability via backward propagation of state probability measures and level-set mappings to a backward value function. It introduces a data-driven procedure that uses forward simulations to empirically estimate these measures and trains a deep neural network to predict reach probabilities for any state and time within a finite horizon, bypassing full dynamic programming. The approach is validated on a 2D double-integrator, a high-dimensional stochastic chain of integrators, and quaternion attitude dynamics, showing competitive accuracy to DP and RKHS baselines and demonstrating favorable scalability with dimension and nonlinear dynamics. The results offer a scalable framework for probabilistic safety guarantees in autonomous systems, with potential extensions to systems with control inputs and finite-action spaces.

Abstract

This paper poses a theoretical characterization of the stochastic reachability problem in terms of probability measures, capturing the probability measure of the state of the system that satisfies the reachability specification for all probabilities over a finite horizon. We achieve this by constructing the level sets of the probability measure for all probability values and, since our approach is only for autonomous systems, we can determine the level sets via forward simulations of the system from a point in the state space at some time step in the finite horizon to estimate the reach probability. We devise a training procedure which exploits this forward simulation and employ it to design a deep neural network (DNN) to predict the reach probability provided the current state and time step. We validate the effectiveness of our approach through three examples.

Figures (4)

• Figure 1: The function approximator takes as input the state and time step, then outputs a reach probability value of satisfying the reach specification.
• Figure 2: We compare our approach to the ground truth via dynamic programming (DP) as well as to a reproducing kernel Hilbert space (RKHS) approach thorpe2022state. In the top row, for a Gaussian disturbance with $\sigma^2 = 0.01$, the RKHS (right) outperforms our method (left), matching closely with DP (middle), as the neural network struggles to learn the sharp drop off of probabilities at the boundaries. However, in the second row, when the Gaussian disturbance has $\sigma^2 = 0.1$ variance, our approach (left) is close to the DP solution (middle) in comparison to RKHS (right). Note that we are able to train the neural network with more samples ($M = 5000$ state samples at training time with $L=2000$ to compute $\alpha$ offline) versus the RKHS method ($M = 10,201$ samples at training time). When attempting to use the same number of samples ($M\cdot L = 10,000,000$ samples) for the RKHS approach, we ran out of memory.
• Figure 3: This plot shows that the training time scales with dimension for the neural network when we fix the number of samples during training. For a fixed sample size, but with increasing state dimension, the neural network scales linearly, similarly to RKHS thorpe2019model. Note that increasing the number of epochs increases the slope of the training curve.
• Figure 4: We compare the proposed approach with an empirical estimate via \ref{['eq:StandardBackwardRecursionEmpirical']}, and the RKHS approach. The neural network (Figure \ref{['fig:DNNQuaternion']}) is closer to the empirical estimate (Figure \ref{['fig:EmpiricalQuaternion']}) than the RKHS (Figure \ref{['fig:RKHSQuaternion']}).

Theorems & Definitions (7)

• Definition 1: bertsekas1996stochastic
• Theorem 1
• proof
• Proposition 1
• proof
• Proposition 2
• proof