Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonparametric estimation of conditional probability distributions using a generative approach based on conditional push-forward neural networks

Nicola Rares Franco, Lorenzo Tedesco

TL;DR

This work introduces conditional push-forward neural networks (CPFN), a lightweight, nonparametric generative framework for estimating conditional distributions by learning a stochastic map that samples from Y|X via latent variables. CPFN obviates the need for invertible architectures or adversarial training, instead optimizing a KL-based objective and achieving near-asymptotic consistency with a controllable bias. The approach demonstrates strong empirical performance, often outperforming kernel, tree-based, and other deep learning methods in both univariate and multivariate settings, and shows competitive results on real-world UCI datasets. The results suggest CPFN is a practical, scalable tool for conditional density estimation with broad potential applications in probabilistic forecasting and uncertainty quantification.

Abstract

We introduce conditional push-forward neural networks (CPFN), a generative framework for conditional distribution estimation. Instead of directly modeling the conditional density $f_{Y|X}$, CPFN learns a stochastic map $\varphi=\varphi(x,u)$ such that $\varphi(x,U)$ and $Y|X=x$ follow approximately the same law, with $U$ a suitable random vector of pre-defined latent variables. This enables efficient conditional sampling and straightforward estimation of conditional statistics through Monte Carlo methods. The model is trained via an objective function derived from a Kullback-Leibler formulation, without requiring invertibility or adversarial training. We establish a near-asymptotic consistency result and demonstrate experimentally that CPFN can achieve performance competitive with, or even superior to, state-of-the-art methods, including kernel estimators, tree-based algorithms, and popular deep learning techniques, all while remaining lightweight and easy to train.

Nonparametric estimation of conditional probability distributions using a generative approach based on conditional push-forward neural networks

TL;DR

This work introduces conditional push-forward neural networks (CPFN), a lightweight, nonparametric generative framework for estimating conditional distributions by learning a stochastic map that samples from Y|X via latent variables. CPFN obviates the need for invertible architectures or adversarial training, instead optimizing a KL-based objective and achieving near-asymptotic consistency with a controllable bias. The approach demonstrates strong empirical performance, often outperforming kernel, tree-based, and other deep learning methods in both univariate and multivariate settings, and shows competitive results on real-world UCI datasets. The results suggest CPFN is a practical, scalable tool for conditional density estimation with broad potential applications in probabilistic forecasting and uncertainty quantification.

Abstract

We introduce conditional push-forward neural networks (CPFN), a generative framework for conditional distribution estimation. Instead of directly modeling the conditional density , CPFN learns a stochastic map such that and follow approximately the same law, with a suitable random vector of pre-defined latent variables. This enables efficient conditional sampling and straightforward estimation of conditional statistics through Monte Carlo methods. The model is trained via an objective function derived from a Kullback-Leibler formulation, without requiring invertibility or adversarial training. We establish a near-asymptotic consistency result and demonstrate experimentally that CPFN can achieve performance competitive with, or even superior to, state-of-the-art methods, including kernel estimators, tree-based algorithms, and popular deep learning techniques, all while remaining lightweight and easy to train.

Paper Structure

This paper contains 40 sections, 5 theorems, 107 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Literature review
  3. Our contribution
  4. A note on quantile regression
  5. Outline of the paper
  6. Presentation of the proposed approach
  7. Conditional push-forward neural networks: general idea
  8. Model design and training
  9. Design
  10. Training
  11. Summary
  12. Technical details: data normalization and rescaling
  13. Theoretical analysis
  14. A result of near asymptotic consistency
  15. Simulation studies
  16. ...and 25 more sections

Key Result

Theorem 1

Consider the setting described above. For $M$ large enough, one has for some positive constants $c_1,c_2>0$ independent of $\varepsilon$ and $\delta.$

Figures (3)

  • Figure 1: True vs generated samples of $(X,Y$). Left: true samples, obtained via \ref{['eq:data_process1']}. Right: artificial samples, generated using the trained CPFN. Both samples contain $n = 1000$ observations.
  • Figure 2: True vs predicted conditional density for two different covariate values $X=x_0.$
  • Figure 3: Comparison of conditional distributions in the multivariate test case for three representative covariate values: $x_{\mathrm{ring}}$ (ring-dominant), $x_{\mathrm{trans}}$ (transition), and $x_{\mathrm{blobs}}$ (blob-dominant). Columns correspond to these three settings, while rows (from top to bottom) display: the true conditional density, $1000$ samples from the true conditional distribution, samples generated by the proposed CPFN model, and samples generated by the KCDE. The proposed method accurately reproduces the multimodal ring– and blob–like geometries, while KCDE results appear more diffuse and less faithful to the true distributions.

Theorems & Definitions (22)

  • Remark 1
  • Remark 2
  • Remark 3
  • Theorem 1: Near asymptotic consistency
  • Corollary 1
  • Remark 4: Assumptions discussion
  • proof
  • proof
  • proof
  • proof
  • ...and 12 more