Scalable Importance Sampling in High Dimensions with Low-Rank Mixture Proposals

TL;DR

The paper addresses the challenge of estimating rare-event probabilities in high-dimensional safety-critical systems using importance sampling. It introduces mixtures of probabilistic principal component analyzers (MPPCA) as low-rank, analytically tractable proposal densities whose parameters are learned efficiently via EM, enabling scalable IS with analytical importance weights $p(\\mathbf{x})/q(\\mathbf{x})$. Through cross-entropy and sequential IS on three high-dimensional simulators (Branches, Duffing oscillator, F-16 GCAS), the authors show that MPPCA-based proposals yield more reliable estimates of the failure probability $P_F$, better mode coverage, and lower total sample counts than full-rank GMMs. The results demonstrate the practical value of low-rank mixtures for safety validation and point to future work on latent-factor selection and trajectory-space metrics.

Abstract

Importance sampling is a Monte Carlo technique for efficiently estimating the likelihood of rare events by biasing the sampling distribution towards the rare event of interest. By drawing weighted samples from a learned proposal distribution, importance sampling allows for more sample-efficient estimation of rare events or tails of distributions. A common choice of proposal density is a Gaussian mixture model (GMM). However, estimating full-rank GMM covariance matrices in high dimensions is a challenging task due to numerical instabilities. In this work, we propose using mixtures of probabilistic principal component analyzers (MPPCA) as the parametric proposal density for importance sampling methods. MPPCA models are a type of low-rank mixture model that can be fit quickly using expectation-maximization, even in high-dimensional spaces. We validate our method on three simulated systems, demonstrating consistent gains in sample efficiency and quality of failure distribution characterization.

Figures (2)

• Figure 1: Safety-critical systems often have extremely low failure rates. Failures can arise from a sequence of disturbances, making the problem high-dimensional. While failures can be multimodal, the number of failure modes is typically much lower than the disturbance dimensionality. This motivates our approach of using low-rank mixture proposals to approximate the distribution over disturbances.
• Figure 2: Results for the Branches problem with $d = 40$ (top row), the Duffing oscillator with $d = 100$ (middle row), and the F-16 GCAS system (bottom row). Red samples represent failure events where $f(\vect{x}) \leq 0$, while gray samples represent outcomes where $f(\vect{x}) > 0$. Across all experiments, the ratio of failure samples to non-failure samples is fixed at $1/4$, regardless of the system's failure rate or the effectiveness of the method. The shaded red areas denote the failure regions where the cost function is $\leq 0$.