Functional Autoencoder for Smoothing and Representation Learning

TL;DR

This work addresses the challenge of learning nonlinear representations and smoothing for discretely observed functional data. It introduces the Functional Autoencoder (FAE), which uses a projection-based encoder with functional weights and a basis-based decoder via a coefficient layer to produce smooth functional reconstructions from low-dimensional representations. Through simulations and a real El Niño dataset, FAEs outperform FPCA in nonlinear settings and offer superior smoothing and computational efficiency relative to conventional autoencoders, while preserving compatibility with irregularly spaced observations. The method advances functional data analysis by enabling one-step, unsupervised learning of nonlinear representations and smooth reconstructions, with potential extensions to multivariate functional data and nonlinear functional regression.

Abstract

A common pipeline in functional data analysis is to first convert the discretely observed data to smooth functions, and then represent the functions by a finite-dimensional vector of coefficients summarizing the information. Existing methods for data smoothing and dimensional reduction mainly focus on learning the linear mappings from the data space to the representation space, however, learning only the linear representations may not be sufficient. In this study, we propose to learn the nonlinear representations of functional data using neural network autoencoders designed to process data in the form it is usually collected without the need of preprocessing. We design the encoder to employ a projection layer computing the weighted inner product of the functional data and functional weights over the observed timestamp, and the decoder to apply a recovery layer that maps the finite-dimensional vector extracted from the functional data back to functional space using a set of predetermined basis functions. The developed architecture can accommodate both regularly and irregularly spaced data. Our experiments demonstrate that the proposed method outperforms functional principal component analysis in terms of prediction and classification, and maintains superior smoothing ability and better computational efficiency in comparison to the conventional autoencoders under both linear and nonlinear settings.

Figures (11)

• Figure 1: Functional autoencoder for continuous data with $L=1$ hidden layer.
• Figure 2: Functional autoencoder for discrete data with $L=1$ hidden layer.
• Figure 3: Encoder with a Feature Layer. Notice that the input and feature layers are devoid of parameters at this point and are entirely deterministic given the data and the choice of basis function for $w$.
• Figure 4: Decoder with a Coefficient Layer. Similarly, the last two layers are devoid of parameters and are deterministic.
• Figure 5: A graphical representation of the FAE we propose for discrete functional data. The model represented only has a single hidden layer $h$, that serves the role of latent representation.