Autoencoders

TL;DR

Autoencoders provide a versatile framework for learning compact, informative representations and for generative modeling. The survey covers regularized and variational variants, introduces the reparameterization trick and disentangled representations, and details a wide range of applications from classification to anomaly detection and recommender systems. It also surveys advanced hybrids with GANs, Wasserstein objectives, perceptual losses, and autoregressive decoders, highlighting both opportunities and practical challenges in parameter and architecture design. Overall, the work outlines a landscape where nonlinear compression and generative modeling intersect with scalable inference, pointing to future work on principled latent-structure selection and realism-aware reconstructions.

Abstract

An autoencoder is a specific type of a neural network, which is mainly designed to encode the input into a compressed and meaningful representation, and then decode it back such that the reconstructed input is similar as possible to the original one. This chapter surveys the different types of autoencoders that are mainly used today. It also describes various applications and use-cases of autoencoders.

Figures (8)

• Figure 1: An autoencoder example. The input image is encoded to a compressed representation and then decoded.
• Figure 2: A denoising autoencoder example. The disrupted input image is encoded to a representation and then decoded.
• Figure 3: A Graphical Representatin of VAE
• Figure 4: Generated images of from a variational autoencoder, trained on the MNIST dataset with a prior $p_\mathbf{\theta}(\mathbf{z})=\mathcal{N}(0,\mathcal{I})$. Left: original images from the dataset. Right: generated images.
• Figure 5: An illustration for using autoencoders as regularization for supervised models. Given the reconstruction loss $R(x,\hat{x})$, and the classification lost function $\mathcal{L}(y,\hat{y})$, the new loss function would be $\Tilde{\mathcal{L}}=\mathcal{L}(y,\hat{y})+\lambda R(x,\hat{x})$, where $\lambda$ is the regularization parameter