Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Fast, Robust Elliptical Slice Sampling Implementation for Linearly Truncated Multivariate Normal Distributions

Kaiwen Wu, Jacob R. Gardner

TL;DR

This work tackles efficient sampling from a $d$-dimensional standard normal truncated to a polytope $\\mathcal{D}=\\{ \\mathbf{A}\\mathbf{x} \\le \\mathbf{b}\\}$ via elliptical slice sampling. It introduces an $\\mathcal{O}(m \\log m)$ algorithm to analytically construct the ellipse–polytope intersection, enabling rejection-free updates that are numerically robust and highly parallelizable on GPUs. The approach yields large practical speedups, achieving over $10\\x$ faster per-iteration performance than a BoTorch baseline in high dimensions, and enabling batched multi-chain sampling. The method includes stability enhancements (interval trimming and selective rejection) and has potential extensions to linear equality constraints and differentiable sampling, broadening its applicability to scalable truncated normal problems.

Abstract

Elliptical slice sampling, when adapted to linearly truncated multivariate normal distributions, is a rejection-free Markov chain Monte Carlo method. At its core, it requires analytically constructing an ellipse-polytope intersection. The main novelty of this paper is an algorithm that computes this intersection in $\mathcal{O}(m \log m)$ time, where $m$ is the number of linear inequality constraints representing the polytope. We show that an implementation based on this algorithm enhances numerical stability, speeds up running time, and is easy to parallelize for launching multiple Markov chains.

A Fast, Robust Elliptical Slice Sampling Implementation for Linearly Truncated Multivariate Normal Distributions

TL;DR

This work tackles efficient sampling from a -dimensional standard normal truncated to a polytope via elliptical slice sampling. It introduces an algorithm to analytically construct the ellipse–polytope intersection, enabling rejection-free updates that are numerically robust and highly parallelizable on GPUs. The approach yields large practical speedups, achieving over faster per-iteration performance than a BoTorch baseline in high dimensions, and enabling batched multi-chain sampling. The method includes stability enhancements (interval trimming and selective rejection) and has potential extensions to linear equality constraints and differentiable sampling, broadening its applicability to scalable truncated normal problems.

Abstract

Elliptical slice sampling, when adapted to linearly truncated multivariate normal distributions, is a rejection-free Markov chain Monte Carlo method. At its core, it requires analytically constructing an ellipse-polytope intersection. The main novelty of this paper is an algorithm that computes this intersection in time, where is the number of linear inequality constraints representing the polytope. We show that an implementation based on this algorithm enhances numerical stability, speeds up running time, and is easy to parallelize for launching multiple Markov chains.
Paper Structure (17 sections, 2 theorems, 33 equations, 4 figures, 3 algorithms)

This paper contains 17 sections, 2 theorems, 33 equations, 4 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Elliptical Slice Sampling for Truncated Normal Sampling
  3. The Obvious Algorithm?
  4. Likelihood Testing
  5. A Simple Method for the Ellipse-Polytope Intersection
  6. General Case
  7. Edge Cases: No Intersection and a Single Intersection
  8. Numerical Stability
  9. Experiments
  10. One Dimensional Truncated Normal Sampling
  11. Accelerate High Dimensional Truncated Normal Sampling
  12. Related Work
  13. Conclusion
  14. Non-Standard Normal Distributions
  15. Roots of the Trigonometry Equation
  16. ...and 2 more sections

Key Result

Proposition 1

Let $\alpha_i < \beta_i$ for all $i \in [m]$ and let $\{\alpha_{i_k}\}_{k=1}^{m}$ be sorted in ascending order as in eq:sorted_left_endpoint. Then, the active intervals $I_\mathrm{act} = \cap_{i=1}^{m} ([)]{[0, \alpha_i] \cup [\beta_i, 2\pi]}$ have an equivalent representation where $\gamma_k = \max\{\beta_{i_1}, \beta_{i_2}, \cdots, \beta_{i_k}\}$ is the cumulative max until $\beta_{i_k}$. We in

Figures (4)

  • Figure 1: An ellipse $\mathbf{x}_t \cos\theta + \boldsymbol \nu _t \sin\theta$ whose angle $\theta \in [0, 2\pi]$ increases counterclockwise. The next iterate $\mathbf{x}_{t+1}$ is sampled from the ellipse-polytope intersection, as shown in red. The intersection consists of two disjoint elliptical arcs. The left one is represented by $[\frac{7}{8}\pi, \frac{9}{8}\pi]$ and the right one is represented by $[0, \frac{1}{8}\pi] \cup [\frac{7}{4}\pi, 2\pi]$.
  • Figure 2: The number of intersections between the ellipse $\mathbf{x} \cos\theta + \boldsymbol \nu \sin\theta$ and a hyperplane $\mathbf{a}^\top \mathbf{x} = b$ depends on the ratio $b / r$, where $r = \sqrt{(\mathbf{a}^\top \mathbf{x}_t)^2 + (\mathbf{a}^\top \boldsymbol \nu _t)^2}$.
  • Figure 3: Draw $10^5$ samples from univariate truncated normal distributions with parallel Markov chains.
  • Figure 4: Running time of drawing $1000$ samples from high dimensional truncated normal distributions.

Theorems & Definitions (4)

  • Proposition 1
  • proof : Proof Sketch
  • Proposition 1
  • proof