Aligning Embeddings and Geometric Random Graphs: Informational Results and Computational Approaches for the Procrustes-Wasserstein Problem

TL;DR

The Ping-Pong algorithm is proposed, alternatively estimating the orthogonal transformation and the relabeling, initialized via a Franke-Wolfe convex relaxation and given sufficient conditions for the method to retrieve the planted signal after one single step.

Abstract

The Procrustes-Wasserstein problem consists in matching two high-dimensional point clouds in an unsupervised setting, and has many applications in natural language processing and computer vision. We consider a planted model with two datasets $X,Y$ that consist of $n$ datapoints in $\mathbb{R}^d$, where $Y$ is a noisy version of $X$, up to an orthogonal transformation and a relabeling of the data points. This setting is related to the graph alignment problem in geometric models. In this work, we focus on the euclidean transport cost between the point clouds as a measure of performance for the alignment. We first establish information-theoretic results, in the high ($d \gg \log n$) and low ($d \ll \log n$) dimensional regimes. We then study computational aspects and propose the Ping-Pong algorithm, alternatively estimating the orthogonal transformation and the relabeling, initialized via a Franke-Wolfe convex relaxation. We give sufficient conditions for the method to retrieve the planted signal after one single step. We provide experimental results to compare the proposed approach with the state-of-the-art method of Grave et al. (2019).

Figures (1)

• Figure 1: Influence of the parameters (dimensions $d$, number of points $n$, and noise level $\sigma$) on the accuracy (in terms of overlap) of three different estimators: the relaxed QAP estimator \ref{['eq:relaxed_QAP']} projected on the set of permutation matrices (blue curve), the output of Alg. \ref{['algo:ping-pong']} (red curve), and the output of grave2019unsupervised's algorithm (purple curve). Each dot corresponds to averaging scores over 10 experiments. \ref{['exp:varying_d']}: $\sigma=0.34,n=100$. \ref{['exp:varying_n']}: $\sigma=0.34,d=5$. \ref{['exp:varying_sigma_lowdim', 'exp:varying_sigma']}: $n=200$, $d=2$ and $d=60$ respectively.

Theorems & Definitions (36)

• Lemma 1: Informal
• Remark 1
• Theorem 1
• Theorem 2
• Lemma 2: From $\hat{Q}$ to $\hat{P}$
• Lemma 3: From $\hat{P}$ to $\hat{Q}$
• Proposition 1
• proof : Proof of \ref{['lemma:equivalence_PW_GGA']}
• Definition 1: $\varepsilon-$nets of $\mathcal{O}(d)$
• Remark 2