On Extended Concentration Inequalities for Fast JL Embeddings of Infinite Sets
Edem Boahen, March T. Boedihardjo, Rafael Chiclana, Mark Iwen
TL;DR
This work tackles the problem of achieving fast Johnson-Lindenstrauss embeddings for infinite subsets of $\mathbb{R}^d$ with embedding dimension $k$ that does not depend on the ambient dimension. It extends the Ailon–Liberty framework by constructing $A=BDHD'$ with a Walsh–Hadamard $H$, 4-wise independent $B$, and random diagonal $D,D'$, and proves a stronger-than-sub-exponential tail bound $\mathbb{P}\{ |\|A x\|_2-1|>t \} \le c_3 \exp(-c_4 k^{2/3} t^{4/3})$ for unit vectors $x$, enabling a chaining-based analysis. A chaining theorem is then established, yielding a d-independent bound on the embedding dimension for infinite sets in terms of covering numbers: $k \ge (C/\epsilon^4)\bigl((\ln(1/p))^{3/4}+\sum_{j\ge0} 2^{-j}(\ln N(S,\|\cdot\|_2,1/2^j))^{3/4}\bigr)^2$, though with suboptimal dependence on the covering numbers. The results demonstrate progress toward d-free embeddings and introduce a tail bound that may be of independent interest, while highlighting the need for tighter control of covering-number terms in future work.
Abstract
The Johnson-Lindenstrauss (JL) lemma allows subsets of a high-dimensional space to be embedded into a lower-dimensional space while approximately preserving all pairwise Euclidean distances. This important result has inspired an extensive literature, with a significant portion dedicated to constructing structured random matrices with fast matrix-vector multiplication algorithms that generate such embeddings for finite point sets. In this paper, we briefly consider fast JL embedding matrices for {\it infinite} subsets of $\mathbb{R}^d$. Prior work in this direction such as \cite{oymak2018isometric, mendelson2023column} has focused on constructing fast JL matrices $HD \in \mathbb{R}^{k \times d}$ by multiplying structured matrices with RIP(-like) properties $H \in \mathbb{R}^{k \times d}$ against a random diagonal matrix $D \in \mathbb{R}^{d \times d}$. However, utilizing RIP(-like) matrices $H$ in this fashion necessarily has the unfortunate side effect that the resulting embedding dimension $k$ must depend on the ambient dimension $d$ no matter how simple the infinite set is that one aims to embed. Motivated by this, we explore an alternate strategy for removing this $d$-dependence from $k$ herein: Extending a concentration inequality proven by Ailon and Liberty \cite{Ailon2008fast} in the hope of later utilizing it in a chaining argument to obtain a near-optimal result for infinite sets. %, and $(ii)$ utilizing a simple secondary Gaussian embedding of an initial fast JL embedding of a given infinite set. Though this strategy ultimately fails to provide the near-optimal embedding dimension we seek, along the way we obtain a stronger-than-sub-exponential extension of the concentration inequality in \cite{Ailon2008fast} which may be of independent interest.