A Unified Latent Space Disentanglement VAE Framework with Robust Disentanglement Effectiveness Evaluation

Abstract

Evaluating and interpreting latent representations, such as variational autoencoders (VAEs), remains a significant challenge for diverse data types, especially when ground-truth generative factors are unknown. To address this, we propose a general framework -- bfVAE -- that unifies several state-of-the-art disentangled VAE approaches and generates effective latent space disentanglement, especially for tabular data. To assess the effectiveness of a VAE disentanglement technique, we propose two procedures - Feature Variance Heterogeneity via Latent Traversal (FVH-LT) and Dirty Block Sparse Regression in Latent Space (DBSR-LS) for disentanglement assessment, along with the latent space disentanglement index (LSDI) which uses the outputs of FVH-LT and DBSR-LS to summarize the overall effectiveness of a VAE disentanglement method without requiring access to or knowledge of the ground-truth generative factors. To the best of our knowledge, these are the first assessment tools to achieve this. FVH-LT and DBSR-LS also enhance latent space interpretability and provide guidance on more efficient content generation. To ensure robust and consistent disentanglement, we develop a greedy alignment strategy (GAS) that mitigates label switching and aligns latent dimensions across runs to obtain aggregated results. We assess the bfVAE framework and validate FVH-LT, DBSR-LS, and LSDI in extensive experiments on tabular and image data. The results suggest that bfVAE surpasses existing disentangled VAE frameworks in terms of disentanglement quality, robustness, achieving a near-zero false discovery rate for informative latent dimensions, that FVH-LT and DBSR-LS reliably uncover semantically meaningful and domain-relevant latent structures, and that LSDI makes an effective overall quantitative summary on disentanglement effectiveness.

Figures (11)

• Figure 1: Flowchart for FVH-LT. $\mathbf{x}_i$ contains input features for the $i$-th observation; $z_k$ is the $k$-th LD and $\tilde{\mathbf{x}}_{kl}$ for $k\!=\!1,\ldots, K$ and $l\!=\!1, \ldots,L$ are generated features from LT of $z_k$; $s_{kj}$ is the sample variance of generated data for the $j$-th feature in the $k$-th LT.
• Figure 2: Conceptual illustration of DBSR-LS. Posterior means $\boldsymbol{\mu}_Z$ learned from the encoder are multi-task regression responses; inputs $\mathbf{x}$ are predictors; sparse regression coefficient matrix $\hat{\mathbf{D}}$ summarizes latent-feature associations.
• Figure 3: The GAS procedure. $d_{rk}$ denotes the prior-posterior KL divergence of $k$-th LD in run $r$; $C_{ij}$ represents the correlation along the $i$-th LD between the reference run $r^*$ and the $j$-th LD in current run $r$, computed from FVH-LT (variance matrix $\mathbf{S}$) or DBSR-LS (coefficient matrix $|\mathbf{D}|$).
• Figure 4: Two degenerate cases resulting in LSDI of $0$. Left: All entries of matrix $\mathbf{A}$ are $0$, indicating the complete absence of informative LDs. Right: A single LD dominates all other LDs across all input features in $\mathbf{A}$, resulting in no disentanglement.
• Figure 5: Comparison between bfVAE vs other disentangled VAE frameworks in FA15. (b) also represents an ablation study as factor-, $\beta$-, vanilla- VAEs are special cases of bfVAE by removing a certain component. Darker cells indicate stronger associations between LDs and input features (cell values not shown due to readability reasons).

Theorems & Definitions (5)

• Definition 1: informative latent dimension
• Definition 2: non-informative latent dimension
• Definition 3: perfectly disentangled latent space
• Definition 4: completely entangled latent space
• Definition 5: LSDI