Johnson-Lindenstrauss Lemma Beyond Euclidean Geometry

TL;DR

This work extends the Johnson-Lindenstrauss lemma to non-Euclidean data by (i) embedding dissimilarities in pseudo-Euclidean space with a $(p,q)$ signature and (ii) representing symmetric hollow dissimilarities as generalized power distances, enabling JL projections with controlled distortions. The first approach yields a JL guarantee where the squared $(p,q)$-distance is preserved up to a multiplicative factor modulated by the ratio to the Euclidean norm, while the second provides a power-distance JL lemma with an additive error term tied to a radius that encodes deviation from Euclidean geometry. Together, these results give fine-grained, geometry-aware guarantees and are backed by experiments on synthetic and real-world datasets that show superior performance relative to classical JL in non-Euclidean settings. The methods broaden the practical utility of dimensionality reduction for a wide class of non-Euclidean dissimilarities, with implications for clustering, similarity learning, and downstream analytics in heterogeneous data sources.

Abstract

The Johnson-Lindenstrauss (JL) lemma is a cornerstone of dimensionality reduction in Euclidean space, but its applicability to non-Euclidean data has remained limited. This paper extends the JL lemma beyond Euclidean geometry to handle general dissimilarity matrices that are prevalent in real-world applications. We present two complementary approaches: First, we show the JL transform can be applied to vectors in pseudo-Euclidean space with signature $(p,q)$, providing theoretical guarantees that depend on the ratio of the $(p, q)$ norm and Euclidean norm of two vectors, measuring the deviation from Euclidean geometry. Second, we prove that any symmetric hollow dissimilarity matrix can be represented as a matrix of generalized power distances, with an additional parameter representing the uncertainty level within the data. In this representation, applying the JL transform yields multiplicative approximation with a controlled additive error term proportional to the deviation from Euclidean geometry. Our theoretical results provide fine-grained performance analysis based on the degree to which the input data deviates from Euclidean geometry, making practical and meaningful reduction in dimensionality accessible to a wider class of data. We validate our approaches on both synthetic and real-world datasets, demonstrating the effectiveness of extending the JL lemma to non-Euclidean settings.

Figures (5)

• Figure 1: Left: power distance from a point $p$ to a ball at $q$ of radius $r_q$; Middle: power distance between two balls at points $p, q$ with radius $r_p$ and $r_q$ respectively; Right: Casey's Theorem.
• Figure 2: The left is for Simplex dataset and right for the Brain dataset. Green points indicate the approximation ratio is within the range and red the opposite. The error bars match the upper and lower bounds given by $(1\pm {\varepsilon}\cdot C_{{ij}})$. We sample 20 pairs of dissimilarities for presentation.
• Figure 3: The left is for MNIST and right for CIFAR-10 dataset. We sampled 100 pairs of dissimilarities and adopt $r/100$ for presentation. Using the original $r$ makes the red line at roughly $y = 40000$, which is very loose.
• Figure 4: Illustrations of Power Distance JL Transform Residual Error
• Figure 5: Illustrations of Pseudo Euclidean JL Transform Multiplicative Error

Theorems & Definitions (18)

• Proposition 1.1: Johnson-Lindenstrauss Lemma
• Theorem 1.2: Fine-grained JL lemma, informal Version of \ref{['thm:jl-pq']}
• Theorem 1.3: Power-distance JL Lemma, informal Version of \ref{['thm:JL-power']}
• Proposition 2.1
• Lemma 2.2
• Lemma 2.2
• Theorem 2.3
• Lemma 2.4
• Corollary 2.5
• Theorem 2.6