Geometric Moment Alignment for Domain Adaptation via Siegel Embeddings

TL;DR

This work addresses covariate-shifted unsupervised domain adaptation by reframing moment alignment in a geometry-aware space. First and second moments are embedded jointly into SPD matrices via Siegel embeddings, enabling distance-based alignment with affine-invariant and Hilbert projective metrics on the SPD manifold. A theoretical bound links the Hilbert distance to target generalization, and the approach yields empirical gains on image denoising and classification benchmarks with compact latent representations. The results indicate that respecting the intrinsic non-Euclidean geometry of moments leads to more faithful cross-domain comparisons and robust adaptation, with code released for reproducibility.

Abstract

We address the problem of distribution shift in unsupervised domain adaptation with a moment-matching approach. Existing methods typically align low-order statistical moments of the source and target distributions in an embedding space using ad-hoc similarity measures. We propose a principled alternative that instead leverages the intrinsic geometry of these distributions by adopting a Riemannian distance for this alignment. Our key novelty lies in expressing the first- and second-order moments as a single symmetric positive definite (SPD) matrix through Siegel embeddings. This enables simultaneous adaptation of both moments using the natural geometric distance on the shared manifold of SPD matrices, preserving the mean and covariance structure of the source and target distributions and yielding a more faithful metric for cross-domain comparison. We connect the Riemannian manifold distance to the target-domain error bound, and validate the method on image denoising and image classification benchmarks. Our code is publicly available at https://github.com/shayangharib/GeoAdapt.

Figures (2)

• Figure 1: $\mathcal{P}$ is the set of all positive-definite matrices endowed with the affine-invariant metric $g_A$. The source and target distributions $\mathbb{P}_S$ and $\mathbb{P}_T$ in the original space are pushed to $\mathcal{P}$ using the embedding $f$ and denoted as $P_S$ and $P_T$ respectively. $\overline{\mathcal{N}}$ (grey area) is a submanifold of $\mathcal{P}$ formed by the projection of Gaussians via $f$. The colored lines conceptually depict paths between them on $\mathcal{P}$: The affine-invariant path is the geodesic path (shortest) in $\mathcal{P}$, the Fisher-Rao path here is the projection by $f$ of the geodesic path on the manifold of Gaussians to $\mathcal{P}$, and the Hilbert projective path is an approximation of the affine-invariant path on $\mathcal{P}$.
• Figure 2: Accuracy on the A$\rightarrow$W setup of the Office-31 dataset as a function of the embedding dimensionality (x-axis). All methods achieve the best accuracy (marked with a point) for dimensionality substantially lower than the full ResNet embedding space. Our distances are the best for the dimensionalities up to our conservative choice of maximum dimensionality where $P_S$ can be robustly inverted.

Theorems & Definitions (14)

• Definition 1
• Remark 1
• Definition 2: Affine-Invariant Riemannian Distance
• Proposition 1
• Remark 2
• Definition 3: Hilbert Projective Distance
• Theorem 1: Upper Bound on Target Error
• Proposition 2
• Definition 4: $\tilde{\mathcal{H}}$-divergence
• Theorem 2