Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Sparse Estimation for Gaussians with Optimal Error under Huber Contamination

Ilias Diakonikolas, Daniel M. Kane, Sushrut Karmalkar, Ankit Pensia, Thanasis Pittas

TL;DR

This work delivers the first computationally efficient procedures for Gaussian sparse estimation under Huber contamination with optimal $O(\epsilon)$ error across mean estimation, sparse PCA, and sparse linear regression. The authors introduce a novel sparse-oriented multidimensional filtering framework that iteratively identifies sparse directions, down-weights outliers, and decomposes the problem into a small coordinate set handled densely, enabling polynomial-time, sample-efficient recovery. They also establish a rigorous certificate-based analysis tying covariance control to tight mean guarantees, and provide reductions that connect robust sparse PCA and regression to robust sparse mean estimation. These results close gaps from previous suboptimal-accuracy methods and set a new baseline for practical robust sparse estimation in high dimensions, with clear avenues for unknown-covariance extensions. Overall, the paper advances the theory and practice of robust statistics in sparse, high-dimensional Gaussian settings, delivering near-optimal error rates with feasible computation.

Abstract

We study Gaussian sparse estimation tasks in Huber's contamination model with a focus on mean estimation, PCA, and linear regression. For each of these tasks, we give the first sample and computationally efficient robust estimators with optimal error guarantees, within constant factors. All prior efficient algorithms for these tasks incur quantitatively suboptimal error. Concretely, for Gaussian robust $k$-sparse mean estimation on $\mathbb{R}^d$ with corruption rate $ε>0$, our algorithm has sample complexity $(k^2/ε^2)\mathrm{polylog}(d/ε)$, runs in sample polynomial time, and approximates the target mean within $\ell_2$-error $O(ε)$. Previous efficient algorithms inherently incur error $Ω(ε\sqrt{\log(1/ε)})$. At the technical level, we develop a novel multidimensional filtering method in the sparse regime that may find other applications.

Robust Sparse Estimation for Gaussians with Optimal Error under Huber Contamination

TL;DR

This work delivers the first computationally efficient procedures for Gaussian sparse estimation under Huber contamination with optimal error across mean estimation, sparse PCA, and sparse linear regression. The authors introduce a novel sparse-oriented multidimensional filtering framework that iteratively identifies sparse directions, down-weights outliers, and decomposes the problem into a small coordinate set handled densely, enabling polynomial-time, sample-efficient recovery. They also establish a rigorous certificate-based analysis tying covariance control to tight mean guarantees, and provide reductions that connect robust sparse PCA and regression to robust sparse mean estimation. These results close gaps from previous suboptimal-accuracy methods and set a new baseline for practical robust sparse estimation in high dimensions, with clear avenues for unknown-covariance extensions. Overall, the paper advances the theory and practice of robust statistics in sparse, high-dimensional Gaussian settings, delivering near-optimal error rates with feasible computation.

Abstract

We study Gaussian sparse estimation tasks in Huber's contamination model with a focus on mean estimation, PCA, and linear regression. For each of these tasks, we give the first sample and computationally efficient robust estimators with optimal error guarantees, within constant factors. All prior efficient algorithms for these tasks incur quantitatively suboptimal error. Concretely, for Gaussian robust -sparse mean estimation on with corruption rate , our algorithm has sample complexity , runs in sample polynomial time, and approximates the target mean within -error . Previous efficient algorithms inherently incur error . At the technical level, we develop a novel multidimensional filtering method in the sparse regime that may find other applications.
Paper Structure (35 sections, 8 theorems, 63 equations, 4 algorithms)

This paper contains 35 sections, 8 theorems, 63 equations, 4 algorithms.

Table of Contents

  1. Introduction
  2. Our Results
  3. Our Techniques
  4. Related Work
  5. Robust PCA
  6. Robust Sparse Estimation
  7. Huber Contamination
  8. Robust Sparse Estimation with Unknown Covariance
  9. Preliminaries
  10. Deterministic Conditions on Inliers
  11. Certificate Lemma
  12. Down-weighting Filter
  13. Robust Sparse Mean Estimation
  14. Algorithm Description and Pseudocode
  15. Proof Overview of \ref{['thm:main']}
  16. ...and 20 more sections

Key Result

Theorem 1.7

For any $\epsilon \in (0,\epsilon_0)$, let $T$ be an $\epsilon$-corrupted set of $n$ samples (in the Huber contamination model) from $\mathcal{N}(\mu,\mathbf I)$ for an unknown $k$-sparse mean $\mu {\in} \mathbb R^d$. There exists an algorithm that, given corruption rate $\epsilon \in (0,\epsilon_0)

Theorems & Definitions (46)

  • Definition 1.1: Huber Contamination Model
  • Definition 1.2: Robust Sparse Mean Estimation
  • Definition 1.4: Robust Sparse PCA
  • Definition 1.5: Robust Sparse Linear Regression
  • Theorem 1.7: Robust Sparse Mean Estimation
  • Theorem 1.8: Robust Sparse PCA
  • Theorem 1.9: Robust Sparse Linear Regression
  • Definition 2.1: Sparse Euclidean Norm
  • Definition 2.2: Sparse Frobenius and Operator Norm
  • proof
  • ...and 36 more