Relative-Translation Invariant Wasserstein Distance

TL;DR

The paper addresses distribution shift under relative translation by introducing ROT and the relative-translation invariant Wasserstein distances $RW_p$, proving $RW_p$ are real metrics on the quotient space $\mathcal{P}_p(\mathbb{R}^n)/\sim$. In the quadratic case, it establishes decomposability, translation-invariance, and a Pythagorean relation with $W_2$, enabling a bias-variance interpretation of shifts. It then develops the $RW_2$ Sinkhorn algorithm for efficient, stable computation of $RW_2$, coupling solutions, and $W_2$. Empirical results in numerical validation, digit recognition under translations, and similar thunderstorm pattern detection demonstrate robustness to translations and notable runtime improvements, highlighting practical applicability in real-world shift scenarios. Overall, the framework provides a principled, scalable approach to comparing distributions when relative translations are present, with broad implications for domain adaptation and pattern recognition under shift.

Abstract

We introduce a new family of distances, relative-translation invariant Wasserstein distances ($RW_p$), for measuring the similarity of two probability distributions under distribution shift. Generalizing it from the classical optimal transport model, we show that $RW_p$ distances are also real distance metrics defined on the quotient set $\mathcal{P}_p(\mathbb{R}^n)/\sim$ and invariant to distribution translations. When $p=2$, the $RW_2$ distance enjoys more exciting properties, including decomposability of the optimal transport model, translation-invariance of the $RW_2$ distance, and a Pythagorean relationship between $RW_2$ and the classical quadratic Wasserstein distance ($W_2$). Based on these properties, we show that a distribution shift, measured by $W_2$ distance, can be explained in the bias-variance perspective. In addition, we propose a variant of the Sinkhorn algorithm, named $RW_2$ Sinkhorn algorithm, for efficiently calculating $RW_2$ distance, coupling solutions, as well as $W_2$ distance. We also provide the analysis of numerical stability and time complexity for the proposed algorithm. Finally, we validate the $RW_2$ distance metric and the algorithm performance with three experiments. We conduct one numerical validation for the $RW_2$ Sinkhorn algorithm and show two real-world applications demonstrating the effectiveness of using $RW_2$ under distribution shift: digits recognition and similar thunderstorm detection. The experimental results report that our proposed algorithm significantly improves the computational efficiency of Sinkhorn in certain practical applications, and the $RW_2$ distance is robust to distribution translations compared with baselines.

Figures (10)

• Figure 1: The relative translation optimal transport problem and $RW_p$ distances.
• Figure 2: Schematic illustration of the first experiment. Assume that the distributions $\mu$ and $\nu$ are the same type of distribution. We compare the performance of Algorithm 1 and the classical Sinkhorn by translating the distribution $\mu$ along vector $s$, i.e., $s =\bar{\nu} - \bar{\mu}$.
• Figure 3: Comparison of the $RW_2$ Sinkhorn algorithm and the classic Sinkhorn in running time and computational error. When the translation is small, the Sinkhorn algorithm with $RW_2$ technique performs similarly or slightly worse than the classical Sinkhorn algorithm. As the translation grows, the Sinkhorn algorithm with $RW_2$ technique significantly outperforms the regular Sinkhorn algorithm.
• Figure 4: Classification accuracy performance of the nearest neighbor search with $RW_2$ distance and other baseline distances ($L_1, L_2, W_1, W_2$) on MNIST dataset. Subfigure (a) shows both the train and test images are perturbed by random translations. Subfigures (b) and (c) show that the $RW_2$ distance is more robust than the other distances against the translational shift.
• Figure 5: Thunderstorm snapshot comparison using $RW_2$ and $W_2$. The leftmost images in the first column are the same reference thunderstorm event. The other images show the top 5 most similar thunderstorm snapshots identified by $RW_2$ and $W_2$, sorted in order of similarity. The $RW_2$ distance focuses more on shape similarity, while the $W_2$ distance pays more attention to location similarity. As observed, the last three images from $W_2$ show much less pattern similarity with the reference.

Theorems & Definitions (24)

• Definition 1: $p$-norm optimal transport problem Villani2009
• Definition 2: Wasserstein distances Villani2009
• Definition 3: Relative translation optimal transport problem
• Theorem 1: Compactness and existence of the minimizer
• Definition 4: Relative-translation invariant Wasserstein distances
• Theorem 2
• Theorem 3: Decomposition of the quadratic ROT
• Corollary 1: Translation-invariance of both the ROT solution and $RW_2$
• Corollary 2: Relationship between $RW_2$ and $W_2$
• proof : Proof of Theorem \ref{['ROT_compact']}