Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Feldman-Hájek Dichotomy for Countable Gaussian Mixtures and their Asymptotic Separability in High Dimensions

Umberto Michelucci

TL;DR

The paper addresses whether Gaussian Mixture Models (GMMs) in high or infinite dimensions can be perfectly separable. It extends the Feldman–Hájek dichotomy to countable mixtures by proving that a countable mixture $\ u(A)=\sum_{k} \pi_k \gamma_k(A)$ is a valid probability measure and that mutual singularity of all component Gaussians implies mutual singularity of the mixtures, formalized in the Gaussian Mixture Dichotomy Theorem. It also analyzes the mixed case via Lebesgue decomposition when some component pairs are equivalent, and discusses the finite vs infinite-dimensional behavior and practical implications for learning. This work provides a theoretical ceiling for zero-error classification in high-dimensional settings and offers a framework for assessing separability and error floors in complex, multi-modal distributions encountered in applications like spectroscopy and anomaly detection.

Abstract

This paper establishes the theoretical foundations for the asymptotic separability of Gaussian Mixture Models (GMMs) in high dimensions by extending the classical Feldman-Hájek theorem. We first prove that a countable mixture of Gaussian measures is a well-defined probability measure. Our primary result, the Gaussian Mixture Dichotomy Theorem, demonstrates that the mutual singularity of individual Gaussian components is a sufficient condition for the mutual singularity of the resulting mixtures. We provide a rigorous proof and further discuss the ``Mixed Case,'' where the presence of even a single equivalent pair of components leads to partial absolute continuity via the Lebesgue decomposition, thereby defining the theoretical limits of perfect classification in infinite-dimensional spaces.

The Feldman-Hájek Dichotomy for Countable Gaussian Mixtures and their Asymptotic Separability in High Dimensions

TL;DR

The paper addresses whether Gaussian Mixture Models (GMMs) in high or infinite dimensions can be perfectly separable. It extends the Feldman–Hájek dichotomy to countable mixtures by proving that a countable mixture is a valid probability measure and that mutual singularity of all component Gaussians implies mutual singularity of the mixtures, formalized in the Gaussian Mixture Dichotomy Theorem. It also analyzes the mixed case via Lebesgue decomposition when some component pairs are equivalent, and discusses the finite vs infinite-dimensional behavior and practical implications for learning. This work provides a theoretical ceiling for zero-error classification in high-dimensional settings and offers a framework for assessing separability and error floors in complex, multi-modal distributions encountered in applications like spectroscopy and anomaly detection.

Abstract

This paper establishes the theoretical foundations for the asymptotic separability of Gaussian Mixture Models (GMMs) in high dimensions by extending the classical Feldman-Hájek theorem. We first prove that a countable mixture of Gaussian measures is a well-defined probability measure. Our primary result, the Gaussian Mixture Dichotomy Theorem, demonstrates that the mutual singularity of individual Gaussian components is a sufficient condition for the mutual singularity of the resulting mixtures. We provide a rigorous proof and further discuss the ``Mixed Case,'' where the presence of even a single equivalent pair of components leads to partial absolute continuity via the Lebesgue decomposition, thereby defining the theoretical limits of perfect classification in infinite-dimensional spaces.
Paper Structure (7 sections, 4 theorems, 17 equations)

This paper contains 7 sections, 4 theorems, 17 equations.

Table of Contents

  1. Introduction
  2. Formalism and Gaussian Mixtures
  3. The Gaussian Mixture Dichotomy Theorem
  4. Results and Discussion
  5. Implications for Statistical Learning
  6. The Mixed Case
  7. Practical Applications

Key Result

Lemma 2.1

Given a Gaussian measure $\gamma_i$, for any For any sequence of disjoint sets $\{A_n\}_{n=1}^\infty$ with $A_n\subset \mathcal{B}(\mathbb{R}^d)$ the sum $\sum_{k=1}^\infty \gamma_i(A_k)$ exists and is finite.

Theorems & Definitions (8)

  • Lemma 2.1
  • proof
  • Lemma 2.2: $\mu$ is a measure
  • proof
  • Lemma 2.3
  • proof
  • Theorem 3.1: The Gaussian Mixture Dichotomy Theorem
  • proof