Phase-Type Variational Autoencoders for Heavy-Tailed Data

TL;DR

This work proposes the Phase-Type Variational Autoencoders (PH-VAE), a flexible and analytically tractable decoder that adapts its tail behavior directly from the observed data, the first work to integrate Phase-Type distributions into deep generative modeling, bridging applied probability and representation learning.

Abstract

Heavy-tailed distributions are ubiquitous in real-world data, where rare but extreme events dominate risk and variability. However, standard Variational Autoencoders (VAEs) employ simple decoder distributions (e.g., Gaussian) that fail to capture heavy-tailed behavior, while existing heavy-tail-aware extensions remain restricted to predefined parametric families whose tail behavior is fixed a priori. We propose the Phase-Type Variational Autoencoder (PH-VAE), whose decoder distribution is a latent-conditioned Phase-Type (PH) distribution defined as the absorption time of a continuous-time Markov chain (CTMC). This formulation composes multiple exponential time scales, yielding a flexible and analytically tractable decoder that adapts its tail behavior directly from the observed data. Experiments on synthetic and real-world benchmarks demonstrate that PH-VAE accurately recovers diverse heavy-tailed distributions, significantly outperforming Gaussian, Student-t, and extreme-value-based VAE decoders in modeling tail behavior and extreme quantiles. In multivariate settings, PH-VAE captures realistic cross-dimensional tail dependence through its shared latent representation. To our knowledge, this is the first work to integrate Phase-Type distributions into deep generative modeling, bridging applied probability and representation learning.

Figures (9)

• Figure 1: Architecture of the Phase-Type Variational Autoencoder (PH-VAE). A shared latent variable $z$ conditions an acyclic Phase-Type decoder, enabling heavy-tailed likelihoods with cross-dimensional dependence.
• Figure 2: A Phase-Type distribution with three transient states.
• Figure 3: Synthetic Weibull data: true samples, Gaussian VAE generations, and PH-VAE generations.
• Figure 4: Log-log CCDF of Danish Fire Insurance losses for real data, Gaussian VAE, and PH-VAE.
• Figure 5: Log--log CCDF of word-frequency counts from the Google Web Trillion Word Corpus, comparing real data, Gaussian VAE, and PH-VAE.