Guessing Efficiently for Constrained Subspace Approximation
Aditya Bhaskara, Sepideh Mahabadi, Madhusudhan Reddy Pittu, Ali Vakilian, David P. Woodruff
TL;DR
This work studies constrained subspace approximation (CSA), seeking a rank-$k$ subspace that minimizes $\|A-PA\|_{2,p}^p$ with projection matrices restricted to a set $\mathcal{S}$. It introduces a generic coreset-guess-solve framework that yields either $(1+\varepsilon)$-multiplicative or $\varepsilon$-additive guarantees across diverse constraint families, by first compressing the data and then solving a reduced, convex subproblem over carefully chosen coefficients or via polynomial-system methods when needed. The framework is applied to PC-$\ell_p$ subspace approximation, constrained subspace estimation, PNMF, $k$-means, and sparse PCA, delivering new or tight bounds and, in many cases, matching state-of-the-art results with running times exponential in $k$ but polynomial in input size for fixed $k$. The paper also establishes hardness for PC-CSS, highlighting intrinsic limits of efficient approximation under partition constraints. Overall, the approach unifies coreset-based strategies with coefficient-guessing and polynomial-system techniques to address a broad class of constrained, non-convex subspace problems.
Abstract
In this paper we study constrained subspace approximation problem. Given a set of $n$ points $\{a_1,\ldots,a_n\}$ in $\mathbb{R}^d$, the goal of the {\em subspace approximation} problem is to find a $k$ dimensional subspace that best approximates the input points. More precisely, for a given $p\geq 1$, we aim to minimize the $p$th power of the $\ell_p$ norm of the error vector $(\|a_1-\bm{P}a_1\|,\ldots,\|a_n-\bm{P}a_n\|)$, where $\bm{P}$ denotes the projection matrix onto the subspace and the norms are Euclidean. In \emph{constrained} subspace approximation (CSA), we additionally have constraints on the projection matrix $\bm{P}$. In its most general form, we require $\bm{P}$ to belong to a given subset $\mathcal{S}$ that is described explicitly or implicitly. We introduce a general framework for constrained subspace approximation. Our approach, that we term coreset-guess-solve, yields either $(1+\varepsilon)$-multiplicative or $\varepsilon$-additive approximations for a variety of constraints. We show that it provides new algorithms for partition-constrained subspace approximation with applications to {\it fair} subspace approximation, $k$-means clustering, and projected non-negative matrix factorization, among others. Specifically, while we reconstruct the best known bounds for $k$-means clustering in Euclidean spaces, we improve the known results for the remainder of the problems.