Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Liouville Flow Importance Sampler

Yifeng Tian, Nishant Panda, Yen Ting Lin

TL;DR

LFIS introduces a pure flow-based sampler that deterministically transports samples from a base distribution to a target via a neural velocity field $v(x,t)$ that satisfies the Generalized Liouville Equation for a prescribed unnormalized density path $\tilde{\rho}_\ast(x,t)$. The velocity field is learned at discrete times by minimizing a least-squares consistency condition; samples along trajectories are assigned weights from accumulated local errors, enabling unbiased estimation of statistics and $\log \mathcal{Z}$ even when the flow is imperfect. The approach yields asymptotically unbiased estimates and demonstrates competitive or state-of-the-art performance on multimodal and high-dimensional benchmarks, including Gaussian mixtures, funnel distributions, and Log-Gaussian Cox processes, while avoiding stochastic kernels. Compared to existing AIS/SMC and diffusion-based methods, LFIS provides a principled, end-to-end flow framework with explicit path evolution, offering robust sampling with potentially lower tuning burden and strong theoretical grounding for importance weight computation.

Abstract

We present the Liouville Flow Importance Sampler (LFIS), an innovative flow-based model for generating samples from unnormalized density functions. LFIS learns a time-dependent velocity field that deterministically transports samples from a simple initial distribution to a complex target distribution, guided by a prescribed path of annealed distributions. The training of LFIS utilizes a unique method that enforces the structure of a derived partial differential equation to neural networks modeling velocity fields. By considering the neural velocity field as an importance sampler, sample weights can be computed through accumulating errors along the sample trajectories driven by neural velocity fields, ensuring unbiased and consistent estimation of statistical quantities. We demonstrate the effectiveness of LFIS through its application to a range of benchmark problems, on many of which LFIS achieved state-of-the-art performance.

Liouville Flow Importance Sampler

TL;DR

LFIS introduces a pure flow-based sampler that deterministically transports samples from a base distribution to a target via a neural velocity field that satisfies the Generalized Liouville Equation for a prescribed unnormalized density path . The velocity field is learned at discrete times by minimizing a least-squares consistency condition; samples along trajectories are assigned weights from accumulated local errors, enabling unbiased estimation of statistics and even when the flow is imperfect. The approach yields asymptotically unbiased estimates and demonstrates competitive or state-of-the-art performance on multimodal and high-dimensional benchmarks, including Gaussian mixtures, funnel distributions, and Log-Gaussian Cox processes, while avoiding stochastic kernels. Compared to existing AIS/SMC and diffusion-based methods, LFIS provides a principled, end-to-end flow framework with explicit path evolution, offering robust sampling with potentially lower tuning burden and strong theoretical grounding for importance weight computation.

Abstract

We present the Liouville Flow Importance Sampler (LFIS), an innovative flow-based model for generating samples from unnormalized density functions. LFIS learns a time-dependent velocity field that deterministically transports samples from a simple initial distribution to a complex target distribution, guided by a prescribed path of annealed distributions. The training of LFIS utilizes a unique method that enforces the structure of a derived partial differential equation to neural networks modeling velocity fields. By considering the neural velocity field as an importance sampler, sample weights can be computed through accumulating errors along the sample trajectories driven by neural velocity fields, ensuring unbiased and consistent estimation of statistical quantities. We demonstrate the effectiveness of LFIS through its application to a range of benchmark problems, on many of which LFIS achieved state-of-the-art performance.
Paper Structure (27 sections, 10 theorems, 53 equations, 7 figures, 15 tables)

This paper contains 27 sections, 10 theorems, 53 equations, 7 figures, 15 tables.

Table of Contents

  1. Background
  2. Related Work
  3. Summary of our contribution
  4. Liouville Flow Importance Sampler
  5. Governing equation of the velocity field
  6. Learning velocity field for sampling
  7. Neural network as an importance sampler
  8. Choice of $\tilde{\rho}_\ast(x,t)$ and the schedule function $\tau(t)$
  9. Numerical Experiments
  10. Testing problems
  11. Results
  12. Discussion
  13. Existence of Velocity Field
  14. Importance Sampling for Imperfect Neural Networks
  15. (Asymptotically) Unbiased Estimators of the Marginalized Likelihood
  16. ...and 12 more sections

Key Result

Theorem 2.1

Suppose $\partial_t \delta \log \tilde{\rho}_\ast \left(x,t\right)$ is square-integrable and the divergence of target score function is absolutely bounded i.e., $\exists M < \infty$ s.t. $\lvert\nabla_x\cdot S_{\ast}(x,t)\rvert < M$ almost everywhere for all $t$. Then, there exists a solution of Eq.

Figures (7)

  • Figure 1: A schematic diagram demonstrating the workflow of the Liouville Flow Importance Sampler.
  • Figure 2: Sampling performance and sample weight distributions for the type-1 problems: (a-h) 2-D Gaussian mixture and (i-p) 10-D funnel distribution. Subfigures (a,i) show the ground-truth PDF contours (marginalized 2-D contour for the funnel distribution). Subfigures (b-g , j-o) compare the generated samples from the ground-truth distribution and different sampling methods. For the funnel distribution, the samples are projected onto $(x_0,x_1)$ plane. Subfigures (h,p) show the log-weight distributions for different sampling methods.
  • Figure 3: Binary MNIST dataset (a-f), decoded LFIS samples (g-l), and decoded image from random latent space samples (m-r).
  • Figure 4: Mode-separated Gaussian mixture with mixed weights for each mode. Three different configures of the mixed weights are considered. Subfigures (a,d,g) show the ground-truth PDFs with the corresponding weights annotated by each mode. The samples generated from the ground-truth PDF and LFIS are shown in subfigures (b,e,h) and (c,f,i), with the sizes of the samples scaled by the sample weights (note that the sample weights are different from the modal wights).
  • Figure 5: The effects of schedule on $\log\hat{\mathcal{Z}}$ estimation at different $T$s and sample quality.
  • ...and 2 more figures

Theorems & Definitions (24)

  • Theorem 2.1
  • Theorem 2.2
  • Theorem 2.3
  • proof
  • Proposition 2.1
  • proof
  • Theorem 2.2
  • proof
  • Remark 2.3
  • Lemma 2.4
  • ...and 14 more