Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Scalable Topological Regularizers

Hiu-Tung Wong, Darrick Lee, Hong Yan

TL;DR

This work tackles the challenge of incorporating topological features into large-scale latent-space learning by introducing Principal Persistence Measures (PPMs) as a scalable topological summary. By pushing forward the persistent homology of many small subsamples and comparing the resulting PPMs with maximum mean discrepancy (MMD) on an extended feature space, the method yields a differentiable topological regularizer that can be integrated into GANs and SSL. The authors prove that MMD-based metrics on PPMs metrize the same topology as Wasserstein distances and establish gradient continuity under smooth densities, enabling stable training. Empirically, PPM-Reg improves shape matching, improves image generation quality on CelebA/AnimeFace with larger gains at higher resolutions, and enhances SSL performance with very few labels, demonstrating scalability and practical impact for topology-aware learning.

Abstract

Latent space matching, which consists of matching distributions of features in latent space, is a crucial component for tasks such as adversarial attacks and defenses, domain adaptation, and generative modelling. Metrics for probability measures, such as Wasserstein and maximum mean discrepancy, are commonly used to quantify the differences between such distributions. However, these are often costly to compute, or do not appropriately take the geometric and topological features of the distributions into consideration. Persistent homology is a tool from topological data analysis which quantifies the multi-scale topological structure of point clouds, and has recently been used as a topological regularizer in learning tasks. However, computation costs preclude larger scale computations, and discontinuities in the gradient lead to unstable training behavior such as in adversarial tasks. We propose the use of principal persistence measures, based on computing the persistent homology of a large number of small subsamples, as a topological regularizer. We provide a parallelized GPU implementation of this regularizer, and prove that gradients are continuous for smooth densities. Furthermore, we demonstrate the efficacy of this regularizer on shape matching, image generation, and semi-supervised learning tasks, opening the door towards a scalable regularizer for topological features.

Towards Scalable Topological Regularizers

TL;DR

This work tackles the challenge of incorporating topological features into large-scale latent-space learning by introducing Principal Persistence Measures (PPMs) as a scalable topological summary. By pushing forward the persistent homology of many small subsamples and comparing the resulting PPMs with maximum mean discrepancy (MMD) on an extended feature space, the method yields a differentiable topological regularizer that can be integrated into GANs and SSL. The authors prove that MMD-based metrics on PPMs metrize the same topology as Wasserstein distances and establish gradient continuity under smooth densities, enabling stable training. Empirically, PPM-Reg improves shape matching, improves image generation quality on CelebA/AnimeFace with larger gains at higher resolutions, and enhances SSL performance with very few labels, demonstrating scalability and practical impact for topology-aware learning.

Abstract

Latent space matching, which consists of matching distributions of features in latent space, is a crucial component for tasks such as adversarial attacks and defenses, domain adaptation, and generative modelling. Metrics for probability measures, such as Wasserstein and maximum mean discrepancy, are commonly used to quantify the differences between such distributions. However, these are often costly to compute, or do not appropriately take the geometric and topological features of the distributions into consideration. Persistent homology is a tool from topological data analysis which quantifies the multi-scale topological structure of point clouds, and has recently been used as a topological regularizer in learning tasks. However, computation costs preclude larger scale computations, and discontinuities in the gradient lead to unstable training behavior such as in adversarial tasks. We propose the use of principal persistence measures, based on computing the persistent homology of a large number of small subsamples, as a topological regularizer. We provide a parallelized GPU implementation of this regularizer, and prove that gradients are continuous for smooth densities. Furthermore, we demonstrate the efficacy of this regularizer on shape matching, image generation, and semi-supervised learning tasks, opening the door towards a scalable regularizer for topological features.
Paper Structure (25 sections, 12 theorems, 51 equations, 11 figures, 11 tables)

This paper contains 25 sections, 12 theorems, 51 equations, 11 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Latent Space Matching in Generative Adversarial Networks
  3. Principal Persistence Measures
  4. Maximum Mean Discrepancy for PPMs
  5. Topological Regularization with PPMs
  6. Experiments and Results
  7. Computational Setup and Implementation Overview
  8. Shape Matching
  9. Unconditional Image Generation
  10. Semi-Supervised Learning
  11. Conclusion
  12. Background on Kernels and MMD
  13. Characteristic Kernels on $\Omega$.
  14. Metrizing Weak Topology
  15. PPM Regularizer has Continuous Gradients
  16. ...and 10 more sections

Key Result

Theorem 1

Let $k: [0,T]^2 \times [0,T]^2 \to \mathbb{R}$ be a kernel which is universal with respect to $C([0,T]^2)$ (or equivalently, characteristic with respect to $\mathcal{P}([0,T]^2)$. Then, $k_\Omega: \Omega \times \Omega \to \mathbb{R}$ is characteristic with respect to $\mathcal{P}(\Omega)$.

Figures (11)

  • Figure 1: An illustration of PH and PPMs. (a) An example point cloud $X$. (b) Snapshots of the Vietoris-Rips filtration $X_\epsilon$ of $X$ at various $\epsilon$. Edges are added between $x_i$ and $x_j$ when $d(x_i, x_j) > \epsilon$ and higher simplices are added when all pairwise distances are greater than $\epsilon$. (c) The dimension $1$ persistence diagram of $X$ in birth-lifetime coordinates. The one point with large lifetime represents the fact that there is a hole in the dataset which persists through multiple scales. (d) An example of a subsampling (in red) of $4 = 2q+2$ points when $q=1$. (e) Snapshots of the Vietoris-Rips filtration of the subsample, where the distances of the bold lines are $t_b$ and $t_d$. (f) The dimension $1$ principal persistence measure of $X$, where the point given by the example subsample is shown in red.
  • Figure 2: Visual example of PPM-Reg in a shape matching experiment using Cramer or MMD as the main loss function. 1st row: Plots of a reference point cloud (in blue) and the initial condition of a random point cloud (in orange). 2nd Row: Plots of 2-Wasserstein distance between 1-dimensional persistent homology between the reference shape and training shape over optimization steps.
  • Figure 3: CMMD (a,d), FDDinov2 (b,e) and WDlatent (c,f) versus training epochs for the AnimeFace (a-c) and CelebA (d-f) dataset. 10K samples are randomly generated to compute distances; moving averages with a window of 5 are used to smooth the values. Distances recorded every 160 epochs.
  • Figure 4: Plots of the shape matching experiment at convergence after 16000 steps. (a) and (e) use only the Cramer loss. (b) and (f) use Cramer + PPM-Reg. (c) and (g) use only the MMD loss. (d) and (h) use MMD + PPM-Reg.
  • Figure 5: Illustrative example of the PPM-Reg in shape matching using MMD with increasing handicap contain. Plotting 2-Wasserstein distance upto 1-dimensional persistent homology between the reference shape and training shape ($PD_{dist}$) over optimization steps. The reference shape of (a) unit circle and (b) union of two intersecting unit circles. $c_\delta$ indicate the strength of the handicap (detail provided in \ref{['apx:IC']}). Showing that the ability to matching topological features as the handicap contains increase.
  • ...and 6 more figures

Theorems & Definitions (23)

  • Theorem 1
  • Theorem 2
  • Remark 1
  • Theorem 3
  • Lemma 1
  • proof
  • Theorem 4
  • proof
  • Corollary 1
  • Theorem 5
  • ...and 13 more