Fast Dimensionality Reduction from $\ell_2$ to $\ell_p$
Rafael Chiclana, Mark Iwen
TL;DR
The paper provides a simple, fast linear embedding from $\ell_2$ to $\ell_p$ for all $p\in[1,2]$ that preserves Euclidean distances up to a factor $1+\varepsilon$ in the target $\ell_p$ norm, with a runtime of $\mathcal{O}(d \log k)$ when the target dimension satisfies $k \le d^{1/4}$. The construction uses a 4-wise independent matrix $A$, independent Rademacher diagonals, and Hadamard transforms in the form $\Psi x = k^{-1/p} \beta_p^{-1} A D_1 H D_2 H D_3 x$, enabling effective dimension reduction with controlled distortion and high probability guarantees. A key technical contribution is the combination of Talagrand concentration and a Berry-Esseen-type bound for the $\ell_p$ norm, together with a Hadamard-based flattening that ensures coordinate mass is sufficiently spread. The work also proves a general lower bound for embeddings into any target norm, showing $k$ must be at least $\Omega( \varepsilon^{-2} \log(\varepsilon^2 n)/\log(1/\varepsilon) )$, matching the Euclidean lower bound up to log factors. This advances fast, structure-friendly dimensionality reduction for $\ell_p$-based applications, including nearest neighbor search and robust learning tasks.
Abstract
The Johnson-Lindenstrauss (JL) lemma is a fundamental result in dimensionality reduction, ensuring that any finite set $X \subseteq \mathbb{R}^d$ can be embedded into a lower-dimensional space $\mathbb{R}^k$ while approximately preserving all pairwise Euclidean distances. In recent years, embeddings that preserve Euclidean distances when measured via the $\ell_1$ norm in the target space have received increasing attention due to their relevance in applications such as nearest neighbor search in high dimensions. A recent breakthrough by Dirksen, Mendelson, and Stollenwerk established an optimal $\ell_2 \to \ell_1$ embedding with computational complexity $O(d \log d)$. In this work, we generalize this direction and propose a simple linear embedding from $\ell_2$ to $\ell_p$ for any $p \in [1,2]$ based on a construction of Ailon and Liberty. Our method achieves a reduced runtime of $O(d \log k)$ when $k \leq d^{1/4}$, improving upon prior runtime results when the target dimension is small. Additionally, we show that for \emph{any norm} $\|\cdot\|$ in the target space, any embedding of $(\mathbb{R}^d, \|\cdot\|_2)$ into $(\mathbb{R}^k, \|\cdot\|)$ with distortion $\varepsilon$ generally requires $k = Ω\big(\varepsilon^{-2} \log(\varepsilon^2 n)/\log(1/\varepsilon)\big)$, matching the optimal bound for the $\ell_2$ case up to a logarithmic factor.