Variational Encoder-Decoders for Learning Latent Representations of Physical Systems

TL;DR

This work introduces a Variational Encoder-Decoder (VED) framework to learn low-dimensional latent representations for high-dimensional input–output mappings in physical systems. By combining a Gaussian-parameterized encoder $q_{\varphi}(z|x)$ and decoder $p_{\theta}(y|z)$ with a variational ELBO objective and additional disentanglement regularization on the aggregate latent distribution, the model achieves accurate reconstructions while producing more independent latent features. Empirical results on a groundwater flow surrogate demonstrate that latent dimensions as small as $r=50$ can capture the essential input–output structure, and that careful tuning of KL-weight $\beta$ and covariance penalty $\lambda$ improves both latent disentanglement and the quality of synthetic data generated from Gaussian noise. The findings highlight a practical pathway for building data-driven, low-dimensional surrogates for complex PDE systems, with implications for uncertainty quantification and efficient Bayesian inference.

Abstract

We present a deep-learning Variational Encoder-Decoder (VED) framework for learning data-driven low-dimensional representations of the relationship between high-dimensional parameters of a physical system and the system's high-dimensional observable response. The framework consists of two deep learning-based probabilistic transformations: An encoder mapping parameters to latent codes and a decoder mapping latent codes to the observable response. The hyperparameters of these transformations are identified by maximizing a variational lower bound on the log-conditional distribution of the observable response given parameters. To promote the disentanglement of latent codes, we equip this variational loss with a penalty on the off-diagonal entries of the aggregate distribution covariance of codes. This regularization penalty encourages the pushforward of a standard Gaussian distribution of latent codes to approximate the marginal distribution of the observable response. Using the proposed framework we successfully model the hydraulic pressure response at observation wells of a groundwater flow model as a function of its discrete log-hydraulic transmissivity field. Compared to the canonical correlation analysis encoding, the VED model achieves a lower-dimensional latent representation, with as low as $r = 50$ latent dimensions without a significant loss of reconstruction accuracy. We explore the impact of regularization on model performance, finding that KL-divergence and covariance regularization improve feature disentanglement in latent space while maintaining reconstruction accuracy. Furthermore, we evaluate the generative capabilities of the regularized model by decoding random Gaussian noise, revealing that tuning both $β$ and $λ$ parameters enhances the quality of the generated observable response data.

Figures (8)

• Figure 1: CCA truncated explained output variance as a function of the latent dimension. The red dashed line shows where we capture $95\%$ of the CCA explained variance ($r = 147$).
• Figure 2: Map2Grid data mapping from an irregular grid to a Cartesian grid.
• Figure 3: Encoder architecture. Blocks represent data shape and arrows represent transformations between data blocks.
• Figure 4: (a) Decoder architecture. (b) Residual block architecture.
• Figure 5: Loss histories over 100 training epochs.