Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CINDES: Classification induced neural density estimator and simulator

Dehao Dai, Jianqing Fan, Yihong Gu, Debarghya Mukherjee

TL;DR

This work addresses high-dimensional density estimation by exploiting unknown low-dimensional structure through a neural, structure-agnostic approach. It introduces Classification induced neural density estimator and simulator (CINDES), which reframes density estimation as a classification problem and couples explicit density estimation with score-based diffusion for efficient implicit sampling. Theoretical guarantees show non-asymptotic convergence bounds and minimax-like rates under low-dimensional factorizable and hierarchical composition structures, with explicit guidance on hyperparameters. Empirical results from extensive simulations and a real data application demonstrate superior performance of CINDES for both unconditional and conditional density estimation and for generating high-quality samples.

Abstract

Neural network-based methods for (un)conditional density estimation have recently gained substantial attention, as various neural density estimators have outperformed classical approaches in real-data experiments. Despite these empirical successes, implementation can be challenging due to the need to ensure non-negativity and unit-mass constraints, and theoretical understanding remains limited. In particular, it is unclear whether such estimators can adaptively achieve faster convergence rates when the underlying density exhibits a low-dimensional structure. This paper addresses these gaps by proposing a structure-agnostic neural density estimator that is (i) straightforward to implement and (ii) provably adaptive, attaining faster rates when the true density admits a low-dimensional composition structure. Another key contribution of our work is to show that the proposed estimator integrates naturally into generative sampling pipelines, most notably score-based diffusion models, where it achieves provably faster convergence when the underlying density is structured. We validate its performance through extensive simulations and a real-data application.

CINDES: Classification induced neural density estimator and simulator

TL;DR

This work addresses high-dimensional density estimation by exploiting unknown low-dimensional structure through a neural, structure-agnostic approach. It introduces Classification induced neural density estimator and simulator (CINDES), which reframes density estimation as a classification problem and couples explicit density estimation with score-based diffusion for efficient implicit sampling. Theoretical guarantees show non-asymptotic convergence bounds and minimax-like rates under low-dimensional factorizable and hierarchical composition structures, with explicit guidance on hyperparameters. Empirical results from extensive simulations and a real data application demonstrate superior performance of CINDES for both unconditional and conditional density estimation and for generating high-quality samples.

Abstract

Neural network-based methods for (un)conditional density estimation have recently gained substantial attention, as various neural density estimators have outperformed classical approaches in real-data experiments. Despite these empirical successes, implementation can be challenging due to the need to ensure non-negativity and unit-mass constraints, and theoretical understanding remains limited. In particular, it is unclear whether such estimators can adaptively achieve faster convergence rates when the underlying density exhibits a low-dimensional structure. This paper addresses these gaps by proposing a structure-agnostic neural density estimator that is (i) straightforward to implement and (ii) provably adaptive, attaining faster rates when the true density admits a low-dimensional composition structure. Another key contribution of our work is to show that the proposed estimator integrates naturally into generative sampling pipelines, most notably score-based diffusion models, where it achieves provably faster convergence when the underlying density is structured. We validate its performance through extensive simulations and a real-data application.

Paper Structure

This paper contains 30 sections, 13 theorems, 147 equations, 1 figure, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Our contribution
  3. Explicit and Implicit Density Estimation
  4. Setup
  5. Background
  6. Deep neural networks
  7. Diffusion model
  8. Our Method
  9. Theory
  10. Examples and Convergence Rates
  11. Low-dimensional factorizable structure
  12. Low-dimensional compositional structures
  13. Simulation Studies
  14. Unconditional density estimation
  15. Conditional density estimation
  16. ...and 15 more sections

Key Result

Theorem 3.1

Assume Conditions cond:regularity and cond:trunc hold. Then for any $n\ge 3$ and $t>0$, the following event occurs with probability at least $1-2e^{-t}$, where $C$ is a constant depending polynomially on $c_2$.

Figures (1)

  • Figure 1: True density and estimated density by different density estimators (CINDES, RFCDE, and MAF) in one trial for two data-generating processes. Density plots were shown on a $100 \times 100$ grid 2D bounded region. Ground-truth densities were shown in the first column. Each row plots results of one data-generating process: (a) Spherical Gaussian mixture; (b) Elliptical Gaussian mixture.

Theorems & Definitions (24)

  • Definition 1
  • Remark 1
  • Definition 2: Deep ReLU network class
  • Remark 2
  • Theorem 3.1: Explicit density estimator
  • Remark 3
  • Remark 4
  • Corollary 3.2
  • Proposition 3.3
  • Remark 5
  • ...and 14 more