Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Criterion for the resemblance between the mother and the model distribution

Yo Sheena

TL;DR

The paper addresses how to quantify closeness between a mother distribution and a model distribution when the model density is intractable, by introducing a discretization-based criterion using the $Hellinger\ distance$ between sample-derived multinomial bins. It grounds the approach in $f$-divergences and their connection to the Bayes error rate, and develops moving-region discretization to form bins from model or sample data, enabling direct comparison without explicit density forms. A key contribution is a concrete, implementable criterion: if $D[\hat m^{(1)}:\hat m^{(2)}] + \frac{p'}{2n_1} + \sqrt{\frac{8p'}{n_2}} < 8\epsilon^2$ given a target Bayes error rate $0.5-\epsilon$, then the distributions are sufficiently close; the framework also provides bias bounds and asymptotic results to guide sample-size choices. The approach is particularly applicable to evaluating complex generative or Bayesian models (e.g., deep generative methods) where exact densities are unavailable, offering a practical, theoretically grounded tool for model evaluation and selection based on distributional closeness.

Abstract

If the probability distribution model aims to approximate the hidden mother distribution, it is imperative to establish a useful criterion for the resemblance between the mother and the model distributions. This study proposes a criterion that measures the Hellinger distance between discretized (quantized) samples from both distributions. Unlike information criteria such as AIC, this criterion does not require the probability density function of the model distribution, which cannot be explicitly obtained for a complicated model such as a deep learning machine. Second, it can draw a positive conclusion (i.e., both distributions are sufficiently close) under a given threshold, whereas a statistical hypothesis test, such as the Kolmogorov-Smirnov test, cannot genuinely lead to a positive conclusion when the hypothesis is accepted. In this study, we establish a reasonable threshold for the criterion deduced from the Bayes error rate and also present the asymptotic bias of the estimator of the criterion. From these results, a reasonable and easy-to-use criterion is established that can be directly calculated from the two sets of samples from both distributions.

Criterion for the resemblance between the mother and the model distribution

TL;DR

The paper addresses how to quantify closeness between a mother distribution and a model distribution when the model density is intractable, by introducing a discretization-based criterion using the between sample-derived multinomial bins. It grounds the approach in -divergences and their connection to the Bayes error rate, and develops moving-region discretization to form bins from model or sample data, enabling direct comparison without explicit density forms. A key contribution is a concrete, implementable criterion: if given a target Bayes error rate , then the distributions are sufficiently close; the framework also provides bias bounds and asymptotic results to guide sample-size choices. The approach is particularly applicable to evaluating complex generative or Bayesian models (e.g., deep generative methods) where exact densities are unavailable, offering a practical, theoretically grounded tool for model evaluation and selection based on distributional closeness.

Abstract

If the probability distribution model aims to approximate the hidden mother distribution, it is imperative to establish a useful criterion for the resemblance between the mother and the model distributions. This study proposes a criterion that measures the Hellinger distance between discretized (quantized) samples from both distributions. Unlike information criteria such as AIC, this criterion does not require the probability density function of the model distribution, which cannot be explicitly obtained for a complicated model such as a deep learning machine. Second, it can draw a positive conclusion (i.e., both distributions are sufficiently close) under a given threshold, whereas a statistical hypothesis test, such as the Kolmogorov-Smirnov test, cannot genuinely lead to a positive conclusion when the hypothesis is accepted. In this study, we establish a reasonable threshold for the criterion deduced from the Bayes error rate and also present the asymptotic bias of the estimator of the criterion. From these results, a reasonable and easy-to-use criterion is established that can be directly calculated from the two sets of samples from both distributions.
Paper Structure (12 sections, 4 theorems, 161 equations, 2 figures, 6 tables)

This paper contains 12 sections, 4 theorems, 161 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. $f$-Divergence
  3. Discretization Methods
  4. Discretization of Scalar $X$
  5. Discretization of Random Vector $\bm{X}$
  6. One Sample Problem
  7. Two Sample Problem
  8. Criterion for Closeness between Mother and Model Distributions
  9. Appendix
  10. Proof of Theorem \ref{['Theo:Error and Div']}
  11. Proof of Theorem \ref{['asymp_risk_moving']}
  12. Complement to Theorem 4

Key Result

Theorem 1

If $D_f[g_1(x) \,| \, g_2(x)] < \delta$, then where

Figures (2)

  • Figure 1: Graph of $t(x) = \Bigl(1-\Bigl(2\delta x(1-x)\Bigr)^{1/2}\Bigr)/2$ with $\delta=0.2,0.5,1.0$
  • Figure 2: Histograms of $\hat{y}$ and $y$

Theorems & Definitions (8)

  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Theorem 4
  • proof