Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the best approximation by finite Gaussian mixtures

Yun Ma, Yihong Wu, Pengkun Yang

TL;DR

This work provides nonasymptotic, rate-optimal characterizations for approximating Gaussian location mixtures by finite mixtures. It introduces a dual strategy: achievable constructions via local moment matching (and Gauss quadrature) yield tight upper bounds, while a novel spectral-analysis framework ties the fundamental limits to the eigenstructure of trigonometric moment matrices, yielding matching lower bounds. The results cover both compactly supported and tail-decaying mixing families, correcting prior Gaussian-specific lower bounds and extending to sub-Weibull tails, moment-constrained classes, and higher-dimensional or more general mixture models. The findings have implications for constellation design in Gaussian channels, nonparametric density estimation, and sieve-based inference, by clarifying how the required number of components scales with accuracy, tail behavior, and dimension. Overall, the paper advances the understanding of the complexity of Gaussian mixtures and provides practical, rate-optimal methods and bounds for finite-mixture approximations.

Abstract

We consider the problem of approximating a general Gaussian location mixture by finite mixtures. The minimum order of finite mixtures that achieve a prescribed accuracy (measured by various $f$-divergences) is determined within constant factors for the family of mixing distributions with compactly support or appropriate assumptions on the tail probability including subgaussian and subexponential. While the upper bound is achieved using the technique of local moment matching, the lower bound is established by relating the best approximation error to the low-rank approximation of certain trigonometric moment matrices, followed by a refined spectral analysis of their minimum eigenvalue. In the case of Gaussian mixing distributions, this result corrects a previous lower bound in [Allerton Conference 48 (2010) 620-628].

On the best approximation by finite Gaussian mixtures

TL;DR

This work provides nonasymptotic, rate-optimal characterizations for approximating Gaussian location mixtures by finite mixtures. It introduces a dual strategy: achievable constructions via local moment matching (and Gauss quadrature) yield tight upper bounds, while a novel spectral-analysis framework ties the fundamental limits to the eigenstructure of trigonometric moment matrices, yielding matching lower bounds. The results cover both compactly supported and tail-decaying mixing families, correcting prior Gaussian-specific lower bounds and extending to sub-Weibull tails, moment-constrained classes, and higher-dimensional or more general mixture models. The findings have implications for constellation design in Gaussian channels, nonparametric density estimation, and sieve-based inference, by clarifying how the required number of components scales with accuracy, tail behavior, and dimension. Overall, the paper advances the understanding of the complexity of Gaussian mixtures and provides practical, rate-optimal methods and bounds for finite-mixture approximations.

Abstract

We consider the problem of approximating a general Gaussian location mixture by finite mixtures. The minimum order of finite mixtures that achieve a prescribed accuracy (measured by various -divergences) is determined within constant factors for the family of mixing distributions with compactly support or appropriate assumptions on the tail probability including subgaussian and subexponential. While the upper bound is achieved using the technique of local moment matching, the lower bound is established by relating the best approximation error to the low-rank approximation of certain trigonometric moment matrices, followed by a refined spectral analysis of their minimum eigenvalue. In the case of Gaussian mixing distributions, this result corrects a previous lower bound in [Allerton Conference 48 (2010) 620-628].
Paper Structure (45 sections, 36 theorems, 201 equations, 1 figure, 2 algorithms)

This paper contains 45 sections, 36 theorems, 201 equations, 1 figure, 2 algorithms.

Table of Contents

  1. Introduction
  2. Main results
  3. Comparison with previous results
  4. Related work
  5. Organization
  6. Notations
  7. Preliminaries
  8. Trigonometric moment matrices
  9. Orthogonal polynomials
  10. Achievability via moment matching
  11. Compactly supported distributions
  12. Distribution families under tail conditions
  13. Converse via spectrum of moment matrices
  14. General framework
  15. Lower bound via trigonometric moments
  16. ...and 30 more sections

Key Result

Theorem 1

Suppose $M\leq \epsilon^{-c_1}$ for some universal constant $0<c_1<\frac{1}{2}$. Then, for $\epsilon\in(0,\frac{1}{2}]$ and $d$ satisfying Assumption as:f-div,

Figures (1)

  • Figure 1: Capacity gap with $\sigma^2=1,10,50$ as $m$ increases.

Theorems & Definitions (70)

  • Theorem 1
  • Theorem 2
  • Remark 1: Gaussian location mixture with general variance
  • Remark 2
  • Lemma 1
  • proof
  • Theorem 3
  • proof
  • Lemma 2
  • proof
  • ...and 60 more