Generative and Nonparametric Approaches for Conditional Distribution Estimation: Methods, Perspectives, and Comparative Evaluations

TL;DR

This work tackles conditional distribution estimation for high- and moderate-dimensional predictors by evaluating four representative approaches: Hall and Yao's single-index dimension-reduction method, FlexCode and DeepCDE basis-expansion techniques, the GCDS generative sampler, and the conditional DDPM diffusion-based sampler. It formalizes a unified evaluation framework and conducts comprehensive simulations across diverse conditional structures, using metrics such as $W_r$ and MSEs for the conditional mean and standard deviation. The study finds that conditional DDPM generally delivers robust distributional accuracy, often outperforming other methods, albeit with slower sampling times, while nonparametric and KL-divergence-based generative methods offer complementary strengths and trade-offs in training stability and scalability. These results provide practical guidance for selecting conditional-distribution estimators in applications with heteroscedasticity, multimodality, or latent structure, and emphasize the importance of aligning method choice with computational constraints and desired inference tasks.

Abstract

The inference of conditional distributions is a fundamental problem in statistics, essential for prediction, uncertainty quantification, and probabilistic modeling. A wide range of methodologies have been developed for this task. This article reviews and compares several representative approaches spanning classical nonparametric methods and modern generative models. We begin with the single-index method of Hall and Yao (2005), which estimates the conditional distribution through a dimension-reducing index and nonparametric smoothing of the resulting one-dimensional cumulative conditional distribution function. We then examine the basis-expansion approaches, including FlexCode (Izbicki and Lee, 2017) and DeepCDE (Dalmasso et al., 2020), which convert conditional density estimation into a set of nonparametric regression problems. In addition, we discuss two recent generative simulation-based methods that leverage modern deep generative architectures: the generative conditional distribution sampler (Zhou et al., 2023) and the conditional denoising diffusion probabilistic model (Fu et al., 2024; Yang et al., 2025). A systematic numerical comparison of these approaches is provided using a unified evaluation framework that ensures fairness and reproducibility. The performance metrics used for the estimated conditional distribution include the mean-squared errors of conditional mean and standard deviation, as well as the Wasserstein distance. We also discuss their flexibility and computational costs, highlighting the distinct advantages and limitations of each approach.

Figures (2)

• Figure 1: True conditional densities for models M1-M5.
• Figure 2: True conditional densities for models M6-M9.