Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Learning Parallel Pancakes with Mostly Uniform Weights

Ilias Diakonikolas, Daniel M. Kane, Sushrut Karmalkar, Jasper C. H. Lee, Thanasis Pittas

TL;DR

The paper investigates learning k-component GMMs in high dimensions under a shared covariance constraint, introducing parallel pancake constructions to probe the limits of SQ-based algorithms. It proves a tight SQ lower bound of $d^{\Omega(\log k)}$ for the uniform-weights case and provides a quasi-polynomial testing upper bound when most weights are uniform with a few arbitrary weights, revealing a nuanced dependence on weight distribution. The key technical approach blends moment matching and design theory (via $t$-designs and Gaussian quadrature), Hermite analysis, and NGCA reductions to derive both lower and upper bounds. The results illuminate the boundary between computational hardness and tractable testing in structured GMM settings and point toward directions for extending to full learning and more general covariances. These insights have implications for algorithm design in high-dimensional mixture modeling with constrained weight and covariance structure.

Abstract

We study the complexity of learning $k$-mixtures of Gaussians ($k$-GMMs) on $\mathbb{R}^d$. This task is known to have complexity $d^{Ω(k)}$ in full generality. To circumvent this exponential lower bound on the number of components, research has focused on learning families of GMMs satisfying additional structural properties. A natural assumption posits that the component weights are not exponentially small and that the components have the same unknown covariance. Recent work gave a $d^{O(\log(1/w_{\min}))}$-time algorithm for this class of GMMs, where $w_{\min}$ is the minimum weight. Our first main result is a Statistical Query (SQ) lower bound showing that this quasi-polynomial upper bound is essentially best possible, even for the special case of uniform weights. Specifically, we show that it is SQ-hard to distinguish between such a mixture and the standard Gaussian. We further explore how the distribution of weights affects the complexity of this task. Our second main result is a quasi-polynomial upper bound for the aforementioned testing task when most of the weights are uniform while a small fraction of the weights are potentially arbitrary.

On Learning Parallel Pancakes with Mostly Uniform Weights

TL;DR

The paper investigates learning k-component GMMs in high dimensions under a shared covariance constraint, introducing parallel pancake constructions to probe the limits of SQ-based algorithms. It proves a tight SQ lower bound of for the uniform-weights case and provides a quasi-polynomial testing upper bound when most weights are uniform with a few arbitrary weights, revealing a nuanced dependence on weight distribution. The key technical approach blends moment matching and design theory (via -designs and Gaussian quadrature), Hermite analysis, and NGCA reductions to derive both lower and upper bounds. The results illuminate the boundary between computational hardness and tractable testing in structured GMM settings and point toward directions for extending to full learning and more general covariances. These insights have implications for algorithm design in high-dimensional mixture modeling with constrained weight and covariance structure.

Abstract

We study the complexity of learning -mixtures of Gaussians (-GMMs) on . This task is known to have complexity in full generality. To circumvent this exponential lower bound on the number of components, research has focused on learning families of GMMs satisfying additional structural properties. A natural assumption posits that the component weights are not exponentially small and that the components have the same unknown covariance. Recent work gave a -time algorithm for this class of GMMs, where is the minimum weight. Our first main result is a Statistical Query (SQ) lower bound showing that this quasi-polynomial upper bound is essentially best possible, even for the special case of uniform weights. Specifically, we show that it is SQ-hard to distinguish between such a mixture and the standard Gaussian. We further explore how the distribution of weights affects the complexity of this task. Our second main result is a quasi-polynomial upper bound for the aforementioned testing task when most of the weights are uniform while a small fraction of the weights are potentially arbitrary.

Paper Structure

This paper contains 31 sections, 19 theorems, 55 equations, 3 algorithms.

Table of Contents

  1. Introduction
  2. Our Results
  3. Remarks
  4. Remarks
  5. Overview of Techniques
  6. Preliminaries
  7. Notation
  8. Hermite Analysis
  9. Probability Facts
  10. Arithmetic Mean-Geometric Mean Inequality
  11. Non-Gaussian Component Analysis
  12. The Uniform Weights Case
  13. Upper bound on numerator
  14. The Mostly Equal Weights Case
  15. Proof Sketch of \ref{['thm:main2']}
  16. ...and 16 more sections

Key Result

Theorem 1.2

Let $C$ be a sufficiently large absolute constant, $k>C$ and $d \geq (\log k \log d)^{2}$ be integers. If we further restrict the alternative hypothesis in def:pancakes to have $w_i = 1/k$ for all $i \in [k]$, any SQ algorithm requires either $2^{d^{\Omega(1)}}$ queries or at least one query of accu

Theorems & Definitions (34)

  • Definition 1.1: STAT Oracle
  • Theorem 1.2: SQ Lower Bound for Uniform Weights
  • Theorem 1.3: Testing Algorithm for Parallel Pancakes
  • proof
  • Proposition 2.4: Theorem 1.5 in diakonikolas2024sq
  • Proposition 3.0
  • Lemma 3.2: Gaussian Quadrature (Lemma 4.3 in DiaKS17)
  • Lemma 3.3
  • proof
  • Lemma 3.4
  • ...and 24 more