Deterministic Coreset for Lp Subspace

TL;DR

This work presents the first iterative, deterministic construction of an $\varepsilon$-coreset for $\ell_p$ subspace embedding that holds for all $p\in[1,\infty)$ with no failure probability. By leveraging an $\ell_p$ Lewis basis and carefully designed potential and barrier functions, the framework maintains a bounded-loss invariant across iterations and selects weighted rows to form a deterministic coreset of size $O\left(d^{\max\{1,p/2\}}/\varepsilon^{2}\right)$. The approach extends Batson–Spielman–Srivastava to general $p$, recovers the $p=2$ case as a special instance, and also applies to deterministic $\ell_p$ regression via an augmented data matrix. These coresets offer tight guarantees and enable deterministic, efficient solutions for high-dimensional $\ell_p$ regression and subspace embedding tasks, with potential streaming adaptations through merge-and-reduce. Overall, the paper significantly advances deterministic data summarization for robust and general $\ell_p$-based problems in regression and subspace approximation.

Abstract

We introduce the first iterative algorithm for constructing a $\varepsilon$-coreset that guarantees deterministic $\ell_p$ subspace embedding for any $p \in [1,\infty)$ and any $\varepsilon > 0$. For a given full rank matrix $\mathbf{X} \in \mathbb{R}^{n \times d}$ where $n \gg d$, $\mathbf{X}' \in \mathbb{R}^{m \times d}$ is an $(\varepsilon,\ell_p)$-subspace embedding of $\mathbf{X}$, if for every $\mathbf{q} \in \mathbb{R}^d$, $(1-\varepsilon)\|\mathbf{Xq}\|_{p}^{p} \leq \|\mathbf{X'q}\|_{p}^{p} \leq (1+\varepsilon)\|\mathbf{Xq}\|_{p}^{p}$. Specifically, in this paper, $\mathbf{X}'$ is a weighted subset of rows of $\mathbf{X}$ which is commonly known in the literature as a coreset. In every iteration, the algorithm ensures that the loss on the maintained set is upper and lower bounded by the loss on the original dataset with appropriate scalings. So, unlike typical coreset guarantees, due to bounded loss, our coreset gives a deterministic guarantee for the $\ell_p$ subspace embedding. For an error parameter $\varepsilon$, our algorithm takes $O(\mathrm{poly}(n,d,\varepsilon^{-1}))$ time and returns a deterministic $\varepsilon$-coreset, for $\ell_p$ subspace embedding whose size is $O\left(\frac{d^{\max\{1,p/2\}}}{\varepsilon^{2}}\right)$. Here, we remove the $\log$ factors in the coreset size, which had been a long-standing open problem. Our coresets are optimal as they are tight with the lower bound. As an application, our coreset can also be used for approximately solving the $\ell_p$ regression problem in a deterministic manner.

Figures (3)

• Figure 1: Potential Functions
• Figure 3: Potential Functions
• Figure 4: At iteration $t-1$, it is ensured that $\Phi^{\pm}_{t-1} \leq \frac{1}{d^{\max\{0,p/2-1\}}}$ (see Lemma \ref{['lem:LpNewPotential']}). Hence, using lemma \ref{['lem:sensitivityLp']} we have $\phi^{\pm}_{t-1} \leq 1$. Now at $t$, it is ensured that $\Phi^{\pm}_{t} \leq \Phi^{\pm}_{t-1} \leq \frac{1}{d^{\max\{0,p/2-1\}}}$. As result we have $\phi^{\pm}_{t} \leq 1$.

Theorems & Definitions (42)

• Theorem 1.1: Informal Version of Theorem \ref{['thm:Lp']}
• Theorem 1.2: Informal Version of Theorem \ref{['thm:LinearLp']}
• Definition 2.1: Sensitivity
• Definition 2.2: $\varepsilon$-coreset
• Lemma 4.1
• proof
• Lemma 4.2
• proof
• Lemma 4.3
• Theorem 4.4: batson2012twice