Koopman Invariants as Drivers of Emergent Time-Series Clustering in Joint-Embedding Predictive Architectures

TL;DR

The paper investigates why Joint-Embedding Predictive Architectures (JEPAs) often cluster time-series representations by underlying dynamical regimes. It develops a Koopman operator-based theory showing that the JEPA predictive objective promotes learning invariant regime-indicator functions, i.e., eigenfunctions with eigenvalue $1$ of the $\\Delta$-step Koopman operator, within a finite mixture of ergodic regimes. Under an idealized setup with a linear predictor and EMA-target tracking, the loss decomposes into a mean-prediction term and an invariant term, and zero loss is achieved when the encoder spans the invariant subspace $\mathcal{V}$ and the predictor acts as the identity on that subspace. Empirically, on synthetic data with 18 regimes, JEPA learns regime-aligned latent clusters and a linear predictor $M$ that is near-identity on the learned subspace, confirming the theory and highlighting JEPA’s potential for interpretable regime identification and anomaly detection in time-series. This work bridges modern self-supervised learning and dynamical-systems theory, offering principled insights to design more robust, interpretable time-series models.

Abstract

Joint-Embedding Predictive Architectures (JEPAs), a powerful class of self-supervised models, exhibit an unexplained ability to cluster time-series data by their underlying dynamical regimes. We propose a novel theoretical explanation for this phenomenon, hypothesizing that JEPA's predictive objective implicitly drives it to learn the invariant subspace of the system's Koopman operator. We prove that an idealized JEPA loss is minimized when the encoder represents the system's regime indicator functions, which are Koopman eigenfunctions. This theory was validated on synthetic data with known dynamics, demonstrating that constraining the JEPA's linear predictor to be a near-identity operator is the key inductive bias that forces the encoder to learn these invariants. We further discuss that this constraint is critical for selecting this interpretable solution from a class of mathematically equivalent but entangled optima, revealing the predictor's role in representation disentanglement. This work demystifies a key behavior of JEPAs, provides a principled connection between modern self-supervised learning and dynamical systems theory, and informs the design of more robust and interpretable time-series models.

Figures (5)

• Figure 1: Conceptual schematic of the Joint-Embedding Predictive Architecture (JEPA). The online encoder and predictor aim to predict the representation of a future target window as generated by the slowly evolving EMA encoder.
• Figure 2: Example waveforms illustrating the diversity of the synthetic dataset.
• Figure 3: Latent space visualization (t-SNE) of test set embeddings. (a) JEPA embeddings often form distinct clusters corresponding to underlying dynamical regimes (indicated by colors). (b) Embeddings from a conventional autoencoder with an identical encoder architecture may not exhibit such clear regime-based separation on the same data.
• Figure 4: Sorted magnitudes of the eigenvalues of $M$, showing dominant eigenvalues near 1.0.
• Figure 5: Action of the learned linear predictor $M$ on regime cluster centroids $c_i$. The small relative error indicates that $M$ largely preserves these regime-representative vectors.

Theorems & Definitions (9)

• Definition 2.1: $\Delta$-Step Koopman Operator
• Lemma 3.2: Properties of Regime Indicators and $\mathcal{V}$
• proof : Proof Intuition
• Theorem 3.4: JEPA Learns Regime Indicators
• proof : Proof Intuition
• Lemma A.1: \ref{['lemma:V_properties_theory']} restated
• proof
• Theorem A.2: \ref{['thm:jepa_learns_V_theory']} restated
• proof