Robust Blockwise Random Pivoting: Fast and Accurate Adaptive Interpolative Decomposition

TL;DR

This work tackles the challenge of constructing fast, accurate, and robust interpolative decompositions (ID) by uniting adaptiveness and randomness through a new method called Robust Blockwise Random Pivoting (RBRP). RBRP introduces robust blockwise filtering to achieve hardware-efficient, rank-adaptive, and ID-revealing skeleton selection, addressing adversarial inputs that degrade plain blockwise methods. The authors also develop interpolation-matrix construction techniques, including ID-revealing approaches and oversampled sketchy ID (OSID) to stabilize and accelerate W, with OSID offering gamma-ID-revealing guarantees under randomized embeddings. Extensive experiments across synthetic adversarial data and real-world datasets demonstrate that RBRP delivers competitive skeleton quality and interpolation accuracy, while maintaining practical runtimes on CPU and GPU architectures. Overall, the paper provides a cohesive framework for fast, accurate, and robust ID that scales well and resists adversarial inputs, with direct implications for data selection, PDE solvers, and model compression tasks.

Abstract

The interpolative decomposition (ID) aims to construct a low-rank approximation formed by a basis consisting of row/column skeletons in the original matrix and a corresponding interpolation matrix. This work explores fast and accurate ID algorithms from comprehensive perspectives for empirical performance, including accuracy in both skeleton selection and interpolation matrix construction, efficiency in terms of asymptotic complexity and hardware efficiency, as well as rank adaptiveness. While many algorithms have been developed to optimize some of these aspects, practical ID algorithms proficient in all aspects remain absent. To fill in the gap, we introduce robust blockwise random pivoting (RBRP) that is asymptotically fast, hardware-efficient, and rank-adaptive, providing accurate skeletons and interpolation matrices comparable to the best existing ID algorithms in practice. Through extensive numerical experiments on various synthetic and natural datasets, we demonstrate the appealing empirical performance of RBRP from the aforementioned perspectives, as well as the robustness of RBRP to adversarial inputs.

Figures (7)

• Figure 1: The relative skeletonization error $\mathcal{E}_{\mathbf{X}}\left(S\right)/\|\mathbf{X}\|_F^2$ and skeleton selection time (excluding interpolation matrix construction) of algorithms in \ref{['tab:abbrev_skeleton_selection']} on the adversarial GMM matrix described in \ref{['subsec:data_matrices']} constructed according to \ref{['ex:gmm_pitfall_plain_bas']}. For the sketchy pivoting methods, we only show the results of SkLUPP, as SkCPQR is known to have similar skeleton complexities as SkLUPP but with higher runtimes dong2021simpler.
• Figure 1: Relative interpolation error and total runtime of ID algorithms on the GMM adversarial input (\ref{['ex:gmm_pitfall_plain_bas']}). Recall from \ref{['fig:id_gmm_rpgp']} that in lack of adaptiveness, methods like SqNorm and BRP tend to suffer from suboptimal skeleton complexities on GMM.
• Figure 2: Relative interpolation error and runtime of ID algorithms on MNIST.
• Figure 3: Relative interpolation error and runtime of ID algorithms on CIFAR-10.
• Figure 4: Relative interpolation error and runtime of ID algorithms on Gaussian-exp.

Theorems & Definitions (19)

• Remark 2.1: Greedy squared-norm pivoting is vulnerable but empirically successful
• Remark 2.2: CPQR is inherently sequential but rank-adaptive
• Example 2.1: Adversarial input for squared-norm sampling
• Proposition 3.1: Sequential random pivoting chen2022randomly
• Remark 3.1: Inefficiency of sequential updates
• Remark 3.2: Sketchy pivoting v.s. random pivoting
• Example 4.1: Pitfall of plain blockwise random pivoting
• Remark 4.1: Robust blockwise filtering
• Remark 4.2: Blockwise greedy pivoting
• Conjecture 4.1: Skeleton complexity of RBRP