List-Decodable Sparse Mean Estimation via Difference-of-Pairs Filtering
Ilias Diakonikolas, Daniel M. Kane, Sushrut Karmalkar, Ankit Pensia, Thanasis Pittas
TL;DR
This work addresses list-decodable sparse mean estimation under adversarial outliers, where an $(1-α)$-corrupted dataset contains an unknown $k$-sparse mean $μ$ with $ loor{αm}$ inliers. It introduces a conceptually simple difference-of-pairs filtering framework, combined with Sum-of-Squares certified moment bounds in $k$-sparse directions, to obtain computationally efficient algorithms. The main result shows that with $m=(t k \log n)^{O(t)} \max(1,M^{-2})/α$ samples, the algorithm runs in time $\mathrm{poly}(m n^t)$ and returns $\hat{μ}$ with error $\|\hat{μ}-μ\|_2 = O_t\left( M^{1/t}/α^{O(1)/t} \right)$ with probability $\Omega(α)$; for Gaussian inliers, it achieves the information-theoretic rate $Θ(\sqrt{\log(1/α)})$ with quasi-polynomial resources. The approach extends to provide SQ and low-degree polynomial lower bounds, illustrating an intrinsic information-computation gap in this setting. Overall, the method yields a simpler, faster route to robust sparse mean estimation and informs the limits of efficient algorithms for list-decodable problems with structured sparsity.
Abstract
We study the problem of list-decodable sparse mean estimation. Specifically, for a parameter $α\in (0, 1/2)$, we are given $m$ points in $\mathbb{R}^n$, $\lfloor αm \rfloor$ of which are i.i.d. samples from a distribution $D$ with unknown $k$-sparse mean $μ$. No assumptions are made on the remaining points, which form the majority of the dataset. The goal is to return a small list of candidates containing a vector $\widehat μ$ such that $\| \widehat μ- μ\|_2$ is small. Prior work had studied the problem of list-decodable mean estimation in the dense setting. In this work, we develop a novel, conceptually simpler technique for list-decodable mean estimation. As the main application of our approach, we provide the first sample and computationally efficient algorithm for list-decodable sparse mean estimation. In particular, for distributions with "certifiably bounded" $t$-th moments in $k$-sparse directions and sufficiently light tails, our algorithm achieves error of $(1/α)^{O(1/t)}$ with sample complexity $m = (k\log(n))^{O(t)}/α$ and running time $\mathrm{poly}(mn^t)$. For the special case of Gaussian inliers, our algorithm achieves the optimal error guarantee of $Θ(\sqrt{\log(1/α)})$ with quasi-polynomial sample and computational complexity. We complement our upper bounds with nearly-matching statistical query and low-degree polynomial testing lower bounds.