Autoencoding Dynamics: Topological Limitations and Capabilities

TL;DR

This work addresses whether low-dimensional dynamical systems on a manifold $M$ can be faithfully represented by autoencoders with latent space $\mathbb{R}^\ell$. It proves strong topological limitations (e.g., when $m>\ell$ there is a global obstruction to perfect round-trips) and, for the equal-dimension static case, establishes large-measure regions on $M$ where ideal encodings exist via contractible neighborhoods and local encoder/decoder pairs, with $L^p$-losses arbitrarily small. Extending to dynamics, the authors show that one can intertwine the true flow on $M$ with a latent flow on $\mathbb{R}^m$ on these regions, and, under Euclidean-covering conditions, achieve large-time intertwinings and linear latent behavior on basins of attraction. The paper also provides a concrete S^1 example with a 1D latent space, detailing a two-phase training procedure and a loss-function design to realize static and dynamic encodings, illustrating practical feasibility alongside the theoretical limits. Overall, the results bridge topology, geometry, and learning to delineate what autoencoders can and cannot achieve for manifold-based dynamical representations and point toward topology-respecting architectures and control-oriented latent dynamics.

Abstract

Given a "data manifold" $M\subset \mathbb{R}^n$ and "latent space" $\mathbb{R}^\ell$, an autoencoder is a pair of continuous maps consisting of an "encoder" $E\colon \mathbb{R}^n\to \mathbb{R}^\ell$ and "decoder" $D\colon \mathbb{R}^\ell\to \mathbb{R}^n$ such that the "round trip" map $D\circ E$ is as close as possible to the identity map $\mbox{id}_M$ on $M$. We present various topological limitations and capabilites inherent to the search for an autoencoder, and describe capabilities for autoencoding dynamical systems having $M$ as an invariant manifold.

Figures (10)

• Figure 1: Encoding manifold dynamics into latent Euclidean space, followed by decoding.
• Figure 2: Original flow when restricted to data manifold. We sampled $\theta$ uniformly, drew the circle $x(\theta)$, and overlaid normalized arrows $f(x(\theta))$ tangential to the circle. A set of particular labeled points $A,\ldots,H$ are marked at their $(\cos\theta,\sin\theta)$ locations for future reference.
• Figure 3: Learned encoder $\phi(\theta)$ with labels A--H.
• Figure 4: Latent vector field $h(\phi)$ with A--H overlays.
• Figure 5: Decoder image $D(\phi)$ in $\mathbb{R}^2$. Small circles how the original points on the unit circle labeled $A\ldots H$, and small squares show the respective decoded points $A'\ldots H'$.

Theorems & Definitions (21)

• Proposition 1
• proof
• Theorem 1
• proof
• Theorem 2: kvalheim2024why
• Theorem 3
• Corollary 1
• Corollary 2
• proof
• Corollary 3