Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actions

TL;DR

Equivariant embeddings are learned from unlabeled group actions by training an encoder to map observations into a latent space where group actions act linearly. The method, EbC, jointly infers an implicit group representation and an alignment with the latent factors, and enforces a content–style separation to yield identifiable equivariant structure. The authors prove an identifiability theorem that recovers the latent space up to a linear transform and recovers the group representation up to similarity, then validate on synthetic groups ($SO(n)$, $O(n)$, $GL(n)$) and a structured visual dataset (infinite dSprites), including a finite product group $R_m \times \mathbb{Z}_n \times \mathbb{Z}_n$, with non-abelian groups. This work provides a practical, encoder-only, general-purpose approach to learning equivariant representations from observations of group actions, with potential impact across computer vision, neuroscience, and biology where perturbations or transformations are observed without explicit group supervision.

Abstract

We propose Equivariance by Contrast (EbC) to learn equivariant embeddings from observation pairs $(\mathbf{y}, g \cdot \mathbf{y})$, where $g$ is drawn from a finite group acting on the data. Our method jointly learns a latent space and a group representation in which group actions correspond to invertible linear maps -- without relying on group-specific inductive biases. We validate our approach on the infinite dSprites dataset with structured transformations defined by the finite group $G:= (R_m \times \mathbb{Z}_n \times \mathbb{Z}_n)$, combining discrete rotations and periodic translations. The resulting embeddings exhibit high-fidelity equivariance, with group operations faithfully reproduced in latent space. On synthetic data, we further validate the approach on the non-abelian orthogonal group $O(n)$ and the general linear group $GL(n)$. We also provide a theoretical proof for identifiability. While broad evaluation across diverse group types on real-world data remains future work, our results constitute the first successful demonstration of general-purpose encoder-only equivariant learning from group action observations alone, including non-trivial non-abelian groups and a product group motivated by modeling affine equivariances in computer vision.

Figures (14)

• Figure 1: Commutative diagram showing data and model for EbC.
• Figure 2: Overview of the approach. Left to right: One batch of observed sample consists of $n+1$ paired samples $\{({\bm{y}}_i, {\bm{y}}_i')\}_{i=0}^{n+1}$ where ${\bm{y}}_i'$ are related to ${\bm{y}}_i$ via an unkown group action $g$ such that ${\bm{y}}_i' = g \cdot {\bm{y}}_i$. An encoder $\phi$ maps these observations into latent space $\{(\hat{{\bm{x}}}_i, \hat{{\bm{x}}}_i')\}_{i=0}^{n+1}$, where $n$ samples are used to estimate a representation $\hat{{\bm{R}}}$ of the group.
• Figure 3: Intuitive explanation of the training objective. Given pairs of samples $({\bm{Y}}, {\bm{Y}}')$ connected by an action $g$, infer the most likely action, and apply it to a novel sample ${\bm{y}}$ with different content. The training objective is contrastive, and aims to select the correct matching sample ${\bm{y}}'$ from a selection of positive/negative samples $S$.
• Figure 4: EbC learns faithful representations of group actions. A, a possible ground-truth latent space for idSprites. Depicted is $\mathbb{Z}_n \times \mathbb{Z}_n$. B, the actual embedding obtained by EbC, plotted for a fixed value of the orientation angle and colored in two views for variations in x- and y- direction. C, evaluation of representation quality and content accuracy across an increasing amount of classes. D, impact of observing the single groups (ind.) vs. a combination at each step in training (com.) across different output dimensions. Number in brackets is the number of samples in ${\bm{Y}}$ per output dim.
• Figure 5: Verification on simulated data. A, traversals through the ground-truth latent space of our synthetic datasets. We manually constructed two orthogonal transformations ${\bm{Q}}_x, {\bm{Q}}_y$ perpendicular to each other to obtain a sphere. B, recovered latent space after non-linear demixing and optimal linear alignment. C, comparisons of EbC (green) against a linear baseline (gray) and an InfoNCE baseline (black) across an increasing number of mixing layers.

Theorems & Definitions (11)

• Theorem 1: Identitiability of the Group Representation; informal
• Corollary 1: Equivariance
• proof
• Definition 1: Data generating process
• Definition 2: Diversity Conditions, adapted from roeder2021linear
• Theorem 1: Identifiability of the Group Representation
• Theorem 2: Generalized Canonical Discriminative Form
• proof
• Lemma 1: Equivariance of implicit group representations
• proof