Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive randomized pivoting and volume sampling

Ethan N. Epperly

TL;DR

This work analyzes adaptive randomized pivoting (ARP) for column subset selection and reveals a fundamental link to volume sampling and projection DPPs. By showing ARP's sample is distributed as $\operatorname{VS}_k(\boldsymbol{Q})$, it transfers active linear-regression results to ARP, yielding near-optimal Frobenius-error guarantees and a clean interpretation of ARP as an interpolative decomposition. The authors then introduce fast ARP implementations based on rejection sampling (RejectionRPQR) and an oversampled sketchy variant (SkARP) using OSID, along with block and stability enhancements. They also establish end-to-end guarantees with sparse embeddings and demonstrate substantial practical speedups and competitive accuracy across experiments, highlighting ARP-family methods as fast and accurate tools for row-subset selection in large-scale settings.

Abstract

Adaptive randomized pivoting (ARP) is a recently proposed and highly effective algorithm for column subset selection. This paper reinterprets the ARP algorithm by drawing connections to the volume sampling distribution and active learning algorithms for linear regression. As consequences, this paper presents new analysis for the ARP algorithm and faster implementations using rejection sampling.

Adaptive randomized pivoting and volume sampling

TL;DR

This work analyzes adaptive randomized pivoting (ARP) for column subset selection and reveals a fundamental link to volume sampling and projection DPPs. By showing ARP's sample is distributed as , it transfers active linear-regression results to ARP, yielding near-optimal Frobenius-error guarantees and a clean interpretation of ARP as an interpolative decomposition. The authors then introduce fast ARP implementations based on rejection sampling (RejectionRPQR) and an oversampled sketchy variant (SkARP) using OSID, along with block and stability enhancements. They also establish end-to-end guarantees with sparse embeddings and demonstrate substantial practical speedups and competitive accuracy across experiments, highlighting ARP-family methods as fast and accurate tools for row-subset selection in large-scale settings.

Abstract

Adaptive randomized pivoting (ARP) is a recently proposed and highly effective algorithm for column subset selection. This paper reinterprets the ARP algorithm by drawing connections to the volume sampling distribution and active learning algorithms for linear regression. As consequences, this paper presents new analysis for the ARP algorithm and faster implementations using rejection sampling.

Paper Structure

This paper contains 17 sections, 6 theorems, 31 equations, 2 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Adaptive randomized pivoting
  3. Contributions and outline
  4. Adaptive randomized pivoting
  5. Connection between ARP and volume sampling
  6. Volume sampling and linear regression
  7. Connection to ARP
  8. Fast ARP by rejection sampling
  9. RejectionRPQR
  10. Efficient block RejectionRPQR
  11. Sketchy adaptive randomized pivoting
  12. End-to-end guarantees
  13. Experiments
  14. Experimental setup
  15. Runtime experiments
  16. ...and 2 more sections

Key Result

Theorem 1.1

\newlabelthm:arp0 The low-rank approximations eq:arp-lra produced by ARP satisfy In particular, if $\boldsymbol{Q}$ consists of the $k$ dominant left singular vectors of $\boldsymbol{A}$, then $\boldsymbol{Q}\boldsymbol{Q}^* \boldsymbol{A} = \mleft\llbracket \boldsymbol{A} \mright\rrbracket_k$ is the optimal rank-$k$ approximation to $\boldsymbol{A}$ and

Figures (2)

  • Figure 1: ARP speed tests. Runtime for five row interpolative decomposition methods on dense (left) and sparse (right) test matrices, described in text.
  • Figure 2: ARP accuracy tests. Relative error for five row interpolative decomposition methods on kernel (left) and genetics (right) test matrices. Lines show mean of 100 trials, and shaded regions show the maximum and minimum errors.

Theorems & Definitions (11)

  • Theorem 1.1: Adaptive randomized pivoting
  • Definition 3.1: Volume sampling and $k$-DPPs
  • Theorem 3.2: Active linear regression by volume sampling
  • Theorem 3.3: Randomly pivoted QR and volume sampling
  • Proof 1: Proof of \ref{['thm:arp']}
  • Definition 6.1: SparseStack
  • Theorem 6.2: Linear algebra with SparseStacks
  • Theorem 6.3: Adaptive randomized pivoting: End-to-end guarantees
  • Proof 2
  • Proposition A.1: Optimality of volume sampling
  • ...and 1 more