Fast Sampling Based Sketches for Tensors

TL;DR

The paper introduces a novel sampling-based sketch framework for two- and three-mode tensors, enabling fast $\ell_0$ sampling and $\ell_1$ embeddings on rank-one tensors with runtimes that scale favorably in the dimension. Central to the approach is the $p$-sample primitive, which achieves fast summation via convolution-based techniques and random mode sign flips, suitable for $q\in\{2,3\}$. The authors provide concrete sketch dimensions and time bounds for both $\ell_0$ sampling and $\ell_1$ embeddings, and connect these to downstream tasks such as $\ell_1$ regression through no-contraction/no-dilation guarantees. They also offer practical constructions of $p$-samples with fast summation (via $A_T$, $B_T$, and $C_T$) and report experimental validation on synthetic tensor data, along with publicly available code. Overall, the work advances fast, scalable tensor sketching and opens avenues for higher-mode extensions and broader applications in regression and optimization.

Abstract

We introduce a new approach for applying sampling-based sketches to two and three mode tensors. We illustrate our technique to construct sketches for the classical problems of $\ell_0$ sampling and producing $\ell_1$ embeddings. In both settings we achieve sketches that can be applied to a rank one tensor in $(\mathbb{R}^d)^{\otimes q}$ (for $q=2,3$) in time scaling with $d$ rather than $d^2$ or $d^3$. Our main idea is a particular sampling construction based on fast convolution which allows us to quickly compute sums over sufficiently random subsets of tensor entries.

Theorems & Definitions (30)

• Theorem 1.1
• Theorem 1.2
• Definition 2.1
• Definition 2.2
• Lemma 2.3
• proof
• Theorem 2.4
• proof
• Lemma 2.5
• proof