Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Topological degree as a discrete diagnostic for disentanglement, with applications to the $Δ$VAE

Mahefa Ratsisetraina Ravelonanosy, Vlado Menkovski, Jacobus W. Portegies

TL;DR

This work introduces the encoder topological degree as a discrete diagnostic for disentanglement in a $Δ$VAE with latent space $\mathcal{S}^2$, and develops an algorithm to compute it via a triangulated, homotopy-preserving construction. Through experiments on a spherical-harmonics dataset generated by $SO(3)$ actions, the authors show that the encoder degree converges to $\pm1$ during training, indicating the learned representation preserves the dataset’s topology up to homotopy, while achieving small LSBD scores for effective disentanglement of the rotational factor. Comparisons with the $S$-VAE reveal favorable topology preservation for $Δ$VAE and competitive discriminative metrics, complemented by a semi-supervised LSBD approach that is essential for this setting. The work provides both a practical topological diagnostic and empirical insights into training dynamics, including extensions to higher-dimensional spheres and discussions on robustness and supervision needs, with code available for reproduction.

Abstract

We investigate the ability of Diffusion Variational Autoencoder ($Δ$VAE) with unit sphere $\mathcal{S}^2$ as latent space to capture topological and geometrical structure and disentangle latent factors in datasets. For this, we introduce a new diagnostic of disentanglement: namely the topological degree of the encoder, which is a map from the data manifold to the latent space. By using tools from homology theory, we derive and implement an algorithm that computes this degree. We use the algorithm to compute the degree of the encoder of models that result from the training procedure. Our experimental results show that the $Δ$VAE achieves relatively small LSBD scores, and that regardless of the degree after initialization, the degree of the encoder after training becomes $-1$ or $+1$, which implies that the resulting encoder is at least homotopic to a homeomorphism.

Topological degree as a discrete diagnostic for disentanglement, with applications to the $Δ$VAE

TL;DR

This work introduces the encoder topological degree as a discrete diagnostic for disentanglement in a VAE with latent space , and develops an algorithm to compute it via a triangulated, homotopy-preserving construction. Through experiments on a spherical-harmonics dataset generated by actions, the authors show that the encoder degree converges to during training, indicating the learned representation preserves the dataset’s topology up to homotopy, while achieving small LSBD scores for effective disentanglement of the rotational factor. Comparisons with the -VAE reveal favorable topology preservation for VAE and competitive discriminative metrics, complemented by a semi-supervised LSBD approach that is essential for this setting. The work provides both a practical topological diagnostic and empirical insights into training dynamics, including extensions to higher-dimensional spheres and discussions on robustness and supervision needs, with code available for reproduction.

Abstract

We investigate the ability of Diffusion Variational Autoencoder (VAE) with unit sphere as latent space to capture topological and geometrical structure and disentangle latent factors in datasets. For this, we introduce a new diagnostic of disentanglement: namely the topological degree of the encoder, which is a map from the data manifold to the latent space. By using tools from homology theory, we derive and implement an algorithm that computes this degree. We use the algorithm to compute the degree of the encoder of models that result from the training procedure. Our experimental results show that the VAE achieves relatively small LSBD scores, and that regardless of the degree after initialization, the degree of the encoder after training becomes or , which implies that the resulting encoder is at least homotopic to a homeomorphism.
Paper Structure (29 sections, 6 theorems, 43 equations, 1 table)

This paper contains 29 sections, 6 theorems, 43 equations, 1 table.

Table of Contents

  1. Introduction
  2. Related works
  3. Topological degree as a diagnostic for disentanglement
  4. Topological degree
  5. Computing the topological degree
  6. Experiments
  7. Data
  8. The models
  9. Training
  10. LSBD score
  11. Further metrics
  12. Experimental results
  13. Evolution of the degree during the training
  14. Discussion
  15. Conclusion
  16. ...and 14 more sections

Key Result

lemma thmcounterlemma

Define $\epsilon := \sqrt{3} \sin(\pi/8) > 0.66$. Let $L_f$ be a Lipschitz constant of $f$, and let $N$ such that for all $n\geq N$ we have for all $\tau \in \mathcal{F}_n$. Then for any $n\geq N$ and for any $\tau\in \mathcal{F}_n$, the image by $g$ of the boundary of $\Delta^{\tau}(\mathbb{D}^2)$ is included in one timezone of $T(3)$ cf. Definition timezone.

Theorems & Definitions (14)

  • remark thmcounterremark
  • definition thmcounterdefinition
  • lemma thmcounterlemma
  • proof
  • lemma thmcounterlemma
  • proof
  • lemma thmcounterlemma
  • proof
  • corollary thmcountercorollary
  • proof
  • ...and 4 more