Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SQ Lower Bounds for Non-Gaussian Component Analysis with Weaker Assumptions

Ilias Diakonikolas, Daniel Kane, Lisheng Ren, Yuxin Sun

TL;DR

This work proves near-optimal SQ lower bounds for NGCA under the moment-matching condition only and obtains near-optimal SQ lower bounds for a range of concrete estimation tasks where existing techniques provide sub-optimal or even vacuous guarantees.

Abstract

We study the complexity of Non-Gaussian Component Analysis (NGCA) in the Statistical Query (SQ) model. Prior work developed a general methodology to prove SQ lower bounds for this task that have been applicable to a wide range of contexts. In particular, it was known that for any univariate distribution $A$ satisfying certain conditions, distinguishing between a standard multivariate Gaussian and a distribution that behaves like $A$ in a random hidden direction and like a standard Gaussian in the orthogonal complement, is SQ-hard. The required conditions were that (1) $A$ matches many low-order moments with the standard univariate Gaussian, and (2) the chi-squared norm of $A$ with respect to the standard Gaussian is finite. While the moment-matching condition is necessary for hardness, the chi-squared condition was only required for technical reasons. In this work, we establish that the latter condition is indeed not necessary. In particular, we prove near-optimal SQ lower bounds for NGCA under the moment-matching condition only. Our result naturally generalizes to the setting of a hidden subspace. Leveraging our general SQ lower bound, we obtain near-optimal SQ lower bounds for a range of concrete estimation tasks where existing techniques provide sub-optimal or even vacuous guarantees.

SQ Lower Bounds for Non-Gaussian Component Analysis with Weaker Assumptions

TL;DR

This work proves near-optimal SQ lower bounds for NGCA under the moment-matching condition only and obtains near-optimal SQ lower bounds for a range of concrete estimation tasks where existing techniques provide sub-optimal or even vacuous guarantees.

Abstract

We study the complexity of Non-Gaussian Component Analysis (NGCA) in the Statistical Query (SQ) model. Prior work developed a general methodology to prove SQ lower bounds for this task that have been applicable to a wide range of contexts. In particular, it was known that for any univariate distribution satisfying certain conditions, distinguishing between a standard multivariate Gaussian and a distribution that behaves like in a random hidden direction and like a standard Gaussian in the orthogonal complement, is SQ-hard. The required conditions were that (1) matches many low-order moments with the standard univariate Gaussian, and (2) the chi-squared norm of with respect to the standard Gaussian is finite. While the moment-matching condition is necessary for hardness, the chi-squared condition was only required for technical reasons. In this work, we establish that the latter condition is indeed not necessary. In particular, we prove near-optimal SQ lower bounds for NGCA under the moment-matching condition only. Our result naturally generalizes to the setting of a hidden subspace. Leveraging our general SQ lower bound, we obtain near-optimal SQ lower bounds for a range of concrete estimation tasks where existing techniques provide sub-optimal or even vacuous guarantees.
Paper Structure (35 sections, 15 theorems, 80 equations)

This paper contains 35 sections, 15 theorems, 80 equations.

Table of Contents

  1. Introduction
  2. SQ Model
  3. Our Results
  4. Main Result
  5. Relation to LLL-based Algorithms for NGCA
  6. Applications
  7. List-decodable Gaussian Mean Estimation
  8. Anti-concentration Detection
  9. Learning Periodic Functions
  10. Technical Overview
  11. Preliminaries
  12. Basics of Hermite Polynomials
  13. SQ-Hardness of NGCA: Proof of Theorem \ref{['thm:main-result']}
  14. Fourier Analysis using Hermite Polynomials
  15. $\|\mathbf{A}_k\|_2$ does not grow too fast:
  16. ...and 20 more sections

Key Result

Theorem 1.5

Let $\lambda\in (0,1)$ and $n,m,d\in \mathbb{N}$ with $d$ even and $m,d\leq n^{\lambda}/\log n$. Let $0<\nu<2$ and $A$ be a distribution on $\mathbb{R}^m$ such that for any polynomial $f:\mathbb{R}^m\to \mathbb{R}$ of degree at most $d$ and $\mathbf{E}_{\mathbf{x}\sim\mathcal{N}_m}[f(\mathbf{x})^2]= Let $0<c<(1-\lambda)/4$ and $n$ be at least a sufficiently large constant depending on $c$. Then an

Theorems & Definitions (41)

  • Definition 1.1: SQ Model
  • Definition 1.2: Hidden-Subspace Distribution
  • Definition 1.3: Hypothesis Testing Version of NGCA
  • Theorem 1.5: Main SQ Lower Bound Result
  • Theorem 1.8: SQ Lower Bound for List-Decoding the Mean
  • Lemma 1.9
  • Theorem 1.10: SQ Lower Bound for AC Detection
  • Definition 1.11
  • Theorem 1.13: SQ Lower Bound for Learning Periodic Functions
  • Definition 2.1: Normalized Hermite Polynomial
  • ...and 31 more