Diffusion-Based Failure Sampling for Evaluating Safety-Critical Autonomous Systems

TL;DR

The paper tackles the challenge of validating safety-critical autonomous systems in high-dimensional settings where failures are rare and multimodal. It introduces Diffusion-based Failure Sampling (DiFS), a conditional denoising diffusion model that learns a disturbance distribution conditioned on robustness and uses an adaptive training loop to increasingly bias samples toward failure. DiFS demonstrates superior fidelity, diversity, and sample efficiency compared with CEM and AST across five validation problems up to 1200 dimensions, including a ground collision avoidance scenario for an F-16. This approach enables more reliable exploration of failure modes, potentially improving safety assurance and robust planning in complex autonomous systems.

Abstract

Validating safety-critical autonomous systems in high-dimensional domains such as robotics presents a significant challenge. Existing black-box approaches based on Markov chain Monte Carlo may require an enormous number of samples, while methods based on importance sampling often rely on simple parametric families that may struggle to represent the distribution over failures. We propose to sample the distribution over failures using a conditional denoising diffusion model, which has shown success in complex high-dimensional problems such as robotic task planning. We iteratively train a diffusion model to produce state trajectories closer to failure. We demonstrate the effectiveness of our approach on high-dimensional robotic validation tasks, improving sample efficiency and mode coverage compared to existing black-box techniques.

Figures (4)

• Figure 1: Illustration of a toy rare failure event sampling problem. Events occur when samples from the unit Gaussian have $|x_0|\geq 3$ and $x_1\geq3$. The top figure shows the true failure density and Monte Carlo samples. Sampling methods based on learned optimization tend to suffer mode collapse, while simple parametric families struggle to capture complex distributions. Our diffusion-based sampling approach reliably models complex multimodal failure distributions and scales to high dimensional systems.
• Figure 2: Illustration of the proposed approach. We train a conditional denoising diffusion model to generate disturbance trajectories that are added to the simulation of a physical system either as action or observation noise. After ranking the disturbances based on their evaluated robustness in increasing order, we train the next iteration of the disturbance-generating diffusion model based on all the samples that we have previously seen whose robustness is lower than the bottom $\alpha$ quantile of samples from the $(i-1)$-th diffusion model iteration.
• Figure 3: Samples from the ground truth failure distribution and trained methods on the toy, inverted pendulum, and lunar lander problems. Ground truth samples were collected using a long run of Monte Carlo sampling. Failures are shown in red, while safe trajectories are shown in gray. In each problem, our DiFS algorithm captures multimodal failures better than the baselines.
• Figure 4: PCA projection of DiFS disturbances reveals multimodal behavior along the first principal component for both the inverted pendulum and the lunar lander. We also illustrate the eigenvectors corresponding to the principal components, which we refer to as eigendisturbances.