Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-Degree Method Fails to Predict Robust Subspace Recovery

He Jia, Aravindan Vijayaraghavan

TL;DR

The results suggest that the low-degree method and low-degree moments fail to capture algorithms based on anti-concentration, challenging their universality as a predictor of computational barriers.

Abstract

The low-degree polynomial framework has been highly successful in predicting computational versus statistical gaps for high-dimensional problems in average-case analysis and machine learning. This success has led to the low-degree conjecture, which posits that this method captures the power and limitations of efficient algorithms for a wide class of high-dimensional statistical problems. We identify a natural and basic hypothesis testing problem in $\mathbb{R}^n$ which is polynomial time solvable, but for which the low-degree polynomial method fails to predict its computational tractability even up to degree $k=n^{Ω(1)}$. Moreover, the low-degree moments match exactly up to degree $k=O(\sqrt{\log n/\log\log n})$. Our problem is a special case of the well-studied robust subspace recovery problem. The lower bounds suggest that there is no polynomial time algorithm for this problem. In contrast, we give a simple and robust polynomial time algorithm that solves the problem (and noisy variants of it), leveraging anti-concentration properties of the distribution. Our results suggest that the low-degree method and low-degree moments fail to capture algorithms based on anti-concentration, challenging their universality as a predictor of computational barriers.

Low-Degree Method Fails to Predict Robust Subspace Recovery

TL;DR

The results suggest that the low-degree method and low-degree moments fail to capture algorithms based on anti-concentration, challenging their universality as a predictor of computational barriers.

Abstract

The low-degree polynomial framework has been highly successful in predicting computational versus statistical gaps for high-dimensional problems in average-case analysis and machine learning. This success has led to the low-degree conjecture, which posits that this method captures the power and limitations of efficient algorithms for a wide class of high-dimensional statistical problems. We identify a natural and basic hypothesis testing problem in which is polynomial time solvable, but for which the low-degree polynomial method fails to predict its computational tractability even up to degree . Moreover, the low-degree moments match exactly up to degree . Our problem is a special case of the well-studied robust subspace recovery problem. The lower bounds suggest that there is no polynomial time algorithm for this problem. In contrast, we give a simple and robust polynomial time algorithm that solves the problem (and noisy variants of it), leveraging anti-concentration properties of the distribution. Our results suggest that the low-degree method and low-degree moments fail to capture algorithms based on anti-concentration, challenging their universality as a predictor of computational barriers.
Paper Structure (26 sections, 16 theorems, 84 equations, 3 figures)

This paper contains 26 sections, 16 theorems, 84 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Our Result
  3. Algorithms and Noise Tolerance.
  4. Failure of low-degree polynomials in capturing anti-concentration.
  5. New candidate instance for separations
  6. Comparison to Non-Gaussian Component Analysis (NGCA).
  7. Technical Overview
  8. The Moment Matching Construction.
  9. Extending Low-Degree Method Hardness to Polynomial Degrees.
  10. The Simple Robust Algorithms.
  11. Preliminaries
  12. Lifting Moment Matching to the Multi-Sample Setting
  13. Moment Matching.
  14. Hermite Polynomials
  15. Moment Matching Instance
  16. ...and 11 more sections

Key Result

Theorem 2

The two distributions over $\mathbb{R}^n$ given by the NULL model and the family of planted distributions $\{\mathcal{P}_v: v \in \mathbb{S}^{n-1}\}$ parametrized by unit vectors $v \in \mathbb{S}^{n-1}$ as described above, satisfy the following properties:

Figures (3)

  • Figure 1: Comparison between a 2D scale mixture of Gaussians and a 2D standard Gaussian. The radial distribution (norm) under the scale mixture is more anti-concentrated.
  • Figure 2: Noise Tolerant Algorithm for detecting if an $\alpha$ fraction of the points lie close to a $d=O(1)$-dimensional subspace
  • Figure 3: Algorithm for detecting even under additive perturbations if an $\alpha$ fraction of the points lie close to a $d=O(1)$-dimensional subspace

Theorems & Definitions (36)

  • Theorem 2: Moment Matching up to degree $k=\widetilde{O}(\sqrt{\log n})$
  • Lemma 4: Same as Lemma \ref{['lem:anticonc-onesided']}
  • Claim 5
  • Lemma 5
  • proof
  • Definition 6: Scale mixture of Gaussians
  • Theorem 7
  • proof : Proof of Theorem \ref{['thm:moment-matching']}
  • Lemma 7
  • Theorem 8: Carbery-Wright, carbery2001
  • ...and 26 more