Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders

TL;DR

This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels and presents numerical experiments that verify that the proposed model can effectively manage data with multi-class nearby but disjoint manifolds of different classes, overlapping manifolds, and manifolds with non-trivial topology.

Abstract

Autoencoding is a popular method in representation learning. Conventional autoencoders employ symmetric encoding-decoding procedures and a simple Euclidean latent space to detect hidden low-dimensional structures in an unsupervised way. Some modern approaches to novel data generation such as generative adversarial networks askew this symmetry, but still employ a pair of massive networks--one to generate the image and another to judge the images quality based on priors learned from a training set. This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels. Besides enhancing the capability for handling data with complicated topological and geometric structures, the proposed model can successfully differentiate nearby but disjoint manifolds and intersecting manifolds with only a small amount of supervision. Moreover, this model only requires a low-complexity encoding operation, such as a locally defined linear projection. We discuss the approximation power of such networks and derive a bound that essentially depends on the intrinsic dimension of the data manifold rather than the dimension of ambient space. Next we incorporate bounds for the sampling rate of training data need to faithfully represent a given data manifold. We present numerical experiments that verify that the proposed model can effectively manage data with multi-class nearby but disjoint manifolds of different classes, overlapping manifolds, and manifolds with non-trivial topology. Finally, we conclude with some experiments on computer vision and molecular dynamics problems which showcase the efficacy of our methods on real-world data.

Figures (12)

• Figure 1: Represent a function $F:\mathcal{M} \rightarrow \mathbb{R}$ from samples of $F$ on a manifold $\mathcal{M} \subset \mathbb{R}^D$ through charts. We first parameterize $\mathcal{M}$ with an atlas of overlapping charts $\{U_\alpha,\phi_\alpha\}_\alpha^N$, each of which is homeomorphic to $\mathbb{R}^d$. Then the function $F$ can be locally approximated a chart $U_\alpha$ by composing function $f_\alpha:\mathbb{R}^d \rightarrow \mathbb{R}$ with $\phi_\alpha$. That is, for $x \in U_\alpha$, $F(x) \approx f_\alpha( \phi_\alpha (x))$
• Figure 2: Left: Training Data Right: Reconstruction of training data and sampling of new data using a 2D VAE, a 3D VAE and a 2D Chart autoencoder
• Figure 3: A function on the Swiss roll data. (a) Training Data; (b) Function on flattened data; (c) Points generated by 4D autoencoder; (d) Points generated by learning latent representation and function thereon.
• Figure 4: Network architecture. Black lines indicate the flow of manifold data, red indicates function approximation, blue represent chart probabilities. Boxes with gray outlines represent neural modules and colored boxes represent data spaces.
• Figure 5: Geometric intuition of reach.

Theorems & Definitions (17)

• Definition 4.1
• Theorem 4.2
• proof
• Theorem 4.3
• proof
• Theorem 4.4
• proof
• Theorem 4.5
• proof
• Theorem B.1