Differential Similarity in Higher Dimensional Spaces: Theory and Applications

TL;DR

This work extends the differential similarity framework to $n$-dimensional spaces by unifying a Riemannian dissimilarity metric with a drift-diffusion probabilistic model driven by a potential $U(\boldsymbol{x})$, yielding a coordinate system on a Frobenius integral manifold that enables principled, low-dimensional data encoding. The geometric model defines a radial coordinate $\rho$ along $\nabla U$ and angular coordinates $\Theta$, with a globally defined metric $g_{ij}(\boldsymbol{x})$, and leverages Frobenius integrability to ensure a well-posed, interpretable lower-dimensional embedding. Estimation of $\nabla U(\boldsymbol{x})$ from data is achieved via a mean-shift gradient of the kernel density, enabling computation of geodesic curves and coordinate flows; the method is demonstrated on MNIST and CIFAR-10, showing meaningful, structured encodings and clusterable subspaces. The paper further develops practical strategies (ellipsoidal approximations, product and quotient manifolds) to scale to higher dimensions and complex data, and it discusses future directions, including connections to deep generative models and manifold-structured representations. Overall, the approach provides a mathematically grounded, geometry-aware framework for dimensionality reduction and clustering that respects both local and global data structure and offers a path toward integrating diffusion-based and geometric insights into deep learning contexts.

Abstract

This paper presents an extension and an elaboration of the theory of differential similarity, which was originally proposed in arXiv:1401.2411 [cs.LG]. The goal is to develop an algorithm for clustering and coding that combines a geometric model with a probabilistic model in a principled way. For simplicity, the geometric model in the earlier paper was restricted to the three-dimensional case. The present paper removes this restriction, and considers the full $n$-dimensional case. Although the mathematical model is the same, the strategies for computing solutions in the $n$-dimensional case are different, and one of the main purposes of this paper is to develop and analyze these strategies. Another main purpose is to devise techniques for estimating the parameters of the model from sample data, again in $n$ dimensions. We evaluate the solution strategies and the estimation techniques by applying them to two familiar real-world examples: the classical MNIST dataset and the CIFAR-10 dataset.

Figures (30)

• Figure 1: Contour plot for the surface of a curvilinear Gaussian potential.
• Figure 2: Gradient vector field for the curvilinear Gaussian potential: (a) at $z = -10$; (b) at $z = 10$, $z = 0$ and $z = -10$.
• Figure 3: An integral manifold with two global coordinate systems for the curvilinear Gaussian potential.
• Figure 4: The $\rho$ and $\Theta$ coordinate curves for the curvilinear Gaussian potential.
• Figure 5: Projecting data points from the curvilinear Gaussian potential along the $\rho$ coordinate curves to the Frobenius integral manifold.

Theorems & Definitions (3)

• Theorem 1: Frobenius
• Theorem 2
• Theorem 3