Hilbert Curve Projection Distance for Distribution Comparison

TL;DR

This work introduces the Hilbert Curve Projection (HCP) distance as a scalable surrogate for the Wasserstein distance to compare probability distributions with bounded supports. By projecting distributions along a Hilbert space-filling curve and computing a coupling in the original space, the authors prove that $\mathrm{HCP}_p$ is a metric and satisfies $\mathrm{W}_p(\mu,\nu)\leq \mathrm{HCP}_p(\mu,\nu)$, providing a strong theoretical foundation. To address the curse of dimensionality, two variants are proposed: integral projection robust HCP (IPRHCP) and projection robust HCP (PRHCP), along with convergence guarantees and efficient algorithms. Empirical results on synthetic and real data show HCP closely tracks Wasserstein with far lower cost, while PRHCP delivers robust performance in high dimensions, making the approach attractive for tasks in data classification and generative modeling.

Abstract

Distribution comparison plays a central role in many machine learning tasks like data classification and generative modeling. In this study, we propose a novel metric, called Hilbert curve projection (HCP) distance, to measure the distance between two probability distributions with low complexity. In particular, we first project two high-dimensional probability distributions using Hilbert curve to obtain a coupling between them, and then calculate the transport distance between these two distributions in the original space, according to the coupling. We show that HCP distance is a proper metric and is well-defined for probability measures with bounded supports. Furthermore, we demonstrate that the modified empirical HCP distance with the $L_p$ cost in the $d$-dimensional space converges to its population counterpart at a rate of no more than $O(n^{-1/2\max\{d,p\}})$. To suppress the curse-of-dimensionality, we also develop two variants of the HCP distance using (learnable) subspace projections. Experiments on both synthetic and real-world data show that our HCP distance works as an effective surrogate of the Wasserstein distance with low complexity and overcomes the drawbacks of the sliced Wasserstein distance.

Figures (11)

• Figure 1: (a) The samples of source and target distributions. (b) Illustrations of various distances with the increase of $\alpha$. (c) Comparisons for various distances on their runtime. The proposed HCP distance provides an effective and efficient surrogate of the Wasserstein distance, which performs similarly and has low computational complexity.
• Figure 2: (a) The $k$-order Hilbert curve, with $k=1,2,3$, in 2D space. (b) The comparison between Hilbert curve projection (HCP) and linear projections.
• Figure 3: An illustration of Algorithm \ref{['alg:hcp']} when $d=k=2$. (a) The source (purple) and target (orange) data points, with corresponding hyper-rectangles and $k$-order Hilbert curves. (b) The projected points along the Hilbert curves. (c) The coupling matrix calculated by the projected points. (d) The HCP distance calculates the distance between the original samples based on the coupling matrix.
• Figure 4: (a) CPU time for generating the $k$-order Hilbert curve versus $d$ when $n=100$. (b) Left: HCP distance versus $n$ when $d=2$. Right: CPU time for generating the $k$-order Hilbert curve versus $n$ when $d=2$. (c) Left: HCP distance versus $n$ when $d=10$. Right: CPU time for generating the $k$-order Hilbert curve versus $n$ when $d=10$.
• Figure 5: $\text{PRHCP}_{2,q}(\mu_n,\nu_n)$ versus the dimension $q$ for $q^*=2,4,7,10$.

Theorems & Definitions (33)

• Definition 1: Hilbert Curve Projection Distance
• Remark 1
• Theorem 1
• Theorem 2
• Example 1
• Example 2
• Theorem 3
• Corollary 3.1
• Corollary 3.2
• Corollary 3.3