Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep conditional distribution learning via conditional Föllmer flow

Jinyuan Chang, Zhao Ding, Yuling Jiao, Ruoxuan Li, Jerry Zhijian Yang

TL;DR

This work tackles conditional sampling from high-dimensional distributions by introducing Conditional Föllmer Flow, an ODE-based method that maps a standard Gaussian to the target conditional density via a Lipschitz velocity field on $t\in[0,1)$. It provides a first-principles end-to-end error analysis that aggregates velocity estimation, discretization, and approximation errors, along with concrete rates and guidance for selecting discretization parameters. The method demonstrates strong empirical performance across nonparametric and image-scale tasks, including accurate prediction intervals and high-quality conditional generation, often outperforming established baselines. The approach delivers practical benefits for uncertainty-aware prediction and conditional data synthesis in complex domains, with efficient one-shot sampling after training.

Abstract

We introduce an ordinary differential equation (ODE) based deep generative method for learning conditional distributions, named Conditional Föllmer Flow. Starting from a standard Gaussian distribution, the proposed flow could approximate the target conditional distribution very well when the time is close to 1. For effective implementation, we discretize the flow with Euler's method where we estimate the velocity field nonparametrically using a deep neural network. Furthermore, we also establish the convergence result for the Wasserstein-2 distance between the distribution of the learned samples and the target conditional distribution, providing the first comprehensive end-to-end error analysis for conditional distribution learning via ODE flow. Our numerical experiments showcase its effectiveness across a range of scenarios, from standard nonparametric conditional density estimation problems to more intricate challenges involving image data, illustrating its superiority over various existing conditional density estimation methods.

Deep conditional distribution learning via conditional Föllmer flow

TL;DR

This work tackles conditional sampling from high-dimensional distributions by introducing Conditional Föllmer Flow, an ODE-based method that maps a standard Gaussian to the target conditional density via a Lipschitz velocity field on . It provides a first-principles end-to-end error analysis that aggregates velocity estimation, discretization, and approximation errors, along with concrete rates and guidance for selecting discretization parameters. The method demonstrates strong empirical performance across nonparametric and image-scale tasks, including accurate prediction intervals and high-quality conditional generation, often outperforming established baselines. The approach delivers practical benefits for uncertainty-aware prediction and conditional data synthesis in complex domains, with efficient one-shot sampling after training.

Abstract

We introduce an ordinary differential equation (ODE) based deep generative method for learning conditional distributions, named Conditional Föllmer Flow. Starting from a standard Gaussian distribution, the proposed flow could approximate the target conditional distribution very well when the time is close to 1. For effective implementation, we discretize the flow with Euler's method where we estimate the velocity field nonparametrically using a deep neural network. Furthermore, we also establish the convergence result for the Wasserstein-2 distance between the distribution of the learned samples and the target conditional distribution, providing the first comprehensive end-to-end error analysis for conditional distribution learning via ODE flow. Our numerical experiments showcase its effectiveness across a range of scenarios, from standard nonparametric conditional density estimation problems to more intricate challenges involving image data, illustrating its superiority over various existing conditional density estimation methods.
Paper Structure (36 sections, 14 theorems, 199 equations, 4 figures, 12 tables, 1 algorithm)

This paper contains 36 sections, 14 theorems, 199 equations, 4 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation
  4. Conditional Föllmer Flow
  5. Sampling Procedure
  6. Theoretical Analysis
  7. Numerical Studies
  8. Simulation Study I
  9. Simulation Study II
  10. Real Data Analysis I
  11. Real Data Analysis II
  12. Class Conditional Generation
  13. Image Inpainting
  14. Discussion
  15. Deep Distribution Learning via Föllmer Flow
  16. ...and 21 more sections

Key Result

Theorem 1

Let Assumptions assump:label and assump:bounded support hold. Then, for any $\mathbf{y} \in [0, B]^{d_y}$, the conditional Föllmer flow $(\mathbf{Z}^{\mathbf{y}}_t)_{t \in [0,1)}$ associated to $p_{x\mkern 2mu|\mkern2muy}(\mathbf{x} \mkern 2mu|\mkern2mu \mathbf{y})$ is a unique solution to the ODE s as $t \rightarrow 1$.

Figures (4)

  • Figure 1: Scatter plots of the pairwise samples generated by different methods.
  • Figure 2: MNIST: real images (top-left panel) and generated images for given labels by our proposed method (top-middle panel), Trigonometric (top-right panel), VE-SDE (bottom-left panel), VAE (bottom-middle panel), and WGAN (bottom-right panel).
  • Figure 3: Original testing images $\{\mathbf{X}_i\}_{i=1}^{10}$ (first columns), associated conditions $\{\mathbf{Y}_i\}_{i=1}^{10}$ (second columns, with the covered parts shaded in red), and reconstructed images $\{\mathbf{X}_i^{(j)}\}_{j=1}^{5}$ with $i=1, \ldots, 10$ by the our proposed method, Trigonometric, VE-SDE, VAE and WGAN (from left to right).
  • Figure F1: Scatter plots of the pairwise samples generated under different stopping time $T=0.999$, $0.9995$ and $0.9999$.

Theorems & Definitions (21)

  • Definition 1: Conditional Föllmer Flow
  • Definition 2: Conditional Föllmer Flow Map
  • Theorem 1
  • Remark 1
  • Remark 2
  • Proposition 1
  • Definition 3
  • Remark 3
  • Proposition 2
  • Proposition 3
  • ...and 11 more