Learning Confidence Ellipsoids and Applications to Robust Subspace Recovery
Chao Gao, Liren Shan, Vaidehi Srinivas, Aravindan Vijayaraghavan
TL;DR
This work studies confidence ellipsoids for high-dimensional distributions, addressing the algorithmic challenge of achieving small volume with rigorous coverage. It introduces a polynomial-time method that attains a volume within a factor Γ = β^γ of the best β-conditioned ellipsoid while capturing nearly all inliers, and it shows a near-tight SSE-based hardness barrier. The approach leverages the primal–dual structure of the minimum-volume enclosing ellipsoid, its connection to D-optimal design, and a Brascamp–Lieb-type inequality, combined with an iterative outlier-removal scheme. The results extend to robust subspace recovery, convex confidence sets, and conformal prediction, offering practical tools for uncertainty quantification in worst-case high-dimensional data and highlighting fundamental limits under SSE hardness.
Abstract
We study the problem of finding confidence ellipsoids for an arbitrary distribution in high dimensions. Given samples from a distribution $D$ and a confidence parameter $α$, the goal is to find the smallest volume ellipsoid $E$ which has probability mass $\Pr_{D}[E] \ge 1-α$. Ellipsoids are a highly expressive class of confidence sets as they can capture correlations in the distribution, and can approximate any convex set. This problem has been studied in many different communities. In statistics, this is the classic minimum volume estimator introduced by Rousseeuw as a robust non-parametric estimator of location and scatter. However in high dimensions, it becomes NP-hard to obtain any non-trivial approximation factor in volume when the condition number $β$ of the ellipsoid (ratio of the largest to the smallest axis length) goes to $\infty$. This motivates the focus of our paper: can we efficiently find confidence ellipsoids with volume approximation guarantees when compared to ellipsoids of bounded condition number $β$? Our main result is a polynomial time algorithm that finds an ellipsoid $E$ whose volume is within a $O(β^{γd})$ multiplicative factor of the volume of best $β$-conditioned ellipsoid while covering at least $1-O(α/γ)$ probability mass for any $γ< α$. We complement this with a computational hardness result that shows that such a dependence seems necessary up to constants in the exponent. The algorithm and analysis uses the rich primal-dual structure of the minimum volume enclosing ellipsoid and the geometric Brascamp-Lieb inequality. As a consequence, we obtain the first polynomial time algorithm with approximation guarantees on worst-case instances of the robust subspace recovery problem.