Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PDGMM-VAE: A Variational Autoencoder with Adaptive Per-Dimension Gaussian Mixture Model Priors for Nonlinear ICA

Yuan-Hao Wei, Yan-Jie Sun

Abstract

Independent component analysis is a core framework within blind source separation for recovering latent source signals from observed mixtures under statistical independence assumptions. In this work, we propose PDGMM-VAE, a source-oriented variational autoencoder in which each latent dimension, interpreted explicitly as an individual source signal, is assigned its own Gaussian mixture model prior. Unlike conventional VAE formulations with a shared simple prior, the proposed framework imposes per-dimension heterogeneous prior constraints, enabling the model to capture diverse non-Gaussian source statistics and thereby promote source separation under a probabilistic encoder-decoder architecture. Importantly, the parameters of these per-dimension GMM priors are not fixed in advance, but are adaptively learned and automatically refined toward convergence together with the encoder and decoder parameters under the overall training objective. Within this formulation, the encoder serves as a demixing mapping from observations to latent sources, while the decoder reconstructs the observed mixtures from the inferred components. The proposed model provides a systematic study of an idea that had previously only been noted in our preliminary form, namely, equipping different latent sources with different GMM priors for ICA, and formulates it as a full VAE framework with end-to-end training and per-dimension prior learning. Experimental results on both linear and nonlinear mixing problems demonstrate that PDGMM-VAE can recover latent source signals and achieve satisfactory separation performance.

PDGMM-VAE: A Variational Autoencoder with Adaptive Per-Dimension Gaussian Mixture Model Priors for Nonlinear ICA

Abstract

Independent component analysis is a core framework within blind source separation for recovering latent source signals from observed mixtures under statistical independence assumptions. In this work, we propose PDGMM-VAE, a source-oriented variational autoencoder in which each latent dimension, interpreted explicitly as an individual source signal, is assigned its own Gaussian mixture model prior. Unlike conventional VAE formulations with a shared simple prior, the proposed framework imposes per-dimension heterogeneous prior constraints, enabling the model to capture diverse non-Gaussian source statistics and thereby promote source separation under a probabilistic encoder-decoder architecture. Importantly, the parameters of these per-dimension GMM priors are not fixed in advance, but are adaptively learned and automatically refined toward convergence together with the encoder and decoder parameters under the overall training objective. Within this formulation, the encoder serves as a demixing mapping from observations to latent sources, while the decoder reconstructs the observed mixtures from the inferred components. The proposed model provides a systematic study of an idea that had previously only been noted in our preliminary form, namely, equipping different latent sources with different GMM priors for ICA, and formulates it as a full VAE framework with end-to-end training and per-dimension prior learning. Experimental results on both linear and nonlinear mixing problems demonstrate that PDGMM-VAE can recover latent source signals and achieve satisfactory separation performance.
Paper Structure (22 sections, 54 equations, 11 figures, 1 algorithm)

This paper contains 22 sections, 54 equations, 11 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. ICA, BSS, and nonlinear identifiability
  4. VAE-based nonlinear ICA and generative source models
  5. Gaussian-mixture-prior VAEs for clustering and disentanglement
  6. Relation to our previous work
  7. Methodology
  8. Problem formulation
  9. Generative model
  10. Variational posterior
  11. Reparameterization of latent variables
  12. Per-dimension Gaussian mixture prior
  13. Observation model and reconstruction term
  14. Evidence lower bound
  15. Expanded posterior and prior log-densities
  16. ...and 7 more sections

Figures (11)

  • Figure 1: Illustration of the proposed PDGMM-VAE framework. Each latent dimension is regularized by an adaptive Gaussian mixture prior whose parameters are jointly learned during training.
  • Figure 2: Training curves in the linear ICA experiment, including the total loss, posterior variances, GMM means, GMM variances, GMM weights, and per-source maximum correlations.
  • Figure 3: Comparison between the true sources and the inferred posterior means in the linear ICA experiment after z-score normalization. The shaded regions denote the posterior uncertainty bands.
  • Figure 4: True and estimated source distributions for Source 1 in the linear ICA experiment.
  • Figure 5: True and estimated source distributions for Source 2 in the linear ICA experiment.
  • ...and 6 more figures