Manifold Percolation: from generative model to Reinforce learning

TL;DR

This work reframes generative modeling as a topology problem, introducing continuum percolation and the percolation threshold as observer-centric probes of a model’s data-support geometry. It defines the Percolation Shift, proves scaling relations linking percolation thresholds to manifold volume, and introduces a differentiable topological loss to expand and stabilize the generated support. The approach is demonstrated across diffusion, RL, and language-model settings, showing that topology-aware supervision yields synergistic improvements where fidelity and diversity reinforce each other rather than trade off. By linking geometric connectivity to learning dynamics, the paper offers a unified framework for diagnosing and mitigating implicit mode collapse, with practical implications for long-horizon robustness and policy optimization.

Abstract

Generative modeling is typically framed as learning mapping rules, but from an observer's perspective without access to these rules, the task becomes disentangling the geometric support from the probability distribution. We propose that continuum percolation is uniquely suited to this support analysis, as the sampling process effectively projects high-dimensional density estimation onto a geometric counting problem on the support. In this work, we establish a rigorous correspondence between the topological phase transitions of random geometric graphs and the underlying data manifold in high-dimensional space. By analyzing the relationship between our proposed Percolation Shift metric and FID, we show that this metric captures structural pathologies, such as implicit mode collapse, where standard statistical metrics fail. Finally, we translate this topological phenomenon into a differentiable loss function that guides training. Experimental results confirm that this approach not only prevents manifold shrinkage but also fosters a form of synergistic improvement, where topological stability becomes a prerequisite for sustained high fidelity in both static generation and sequential decision making.

Figures (10)

• Figure 1: Empirical Verification of Manifold Percolation Scaling. We simulate Random Geometric Graphs on hyperspheres ($S^d$) with varying sample sizes $N \in [100, 3000]$. The log-log plot reveals a linear relationship where the slope corresponds precisely to $-1/d$, validating the theoretical scaling law $\varepsilon_c \propto N^{-1/d}$. Error bars denote the standard deviation over 5 independent trials, demonstrating the metric's stability.
• Figure 2: Controlled Experiment on a Toy Manifold. (a) Simulating Implicit Mode Collapse by reducing variance (red) relative to ground truth (gray). (b) The percolation curve shifts left ($\Delta \varepsilon_c < 0$), empirically validating the shrinkage behavior predicted by Theorem \ref{['thm:shrinkage']}.
• Figure 3: The Anatomy of Implicit Mode Collapse. A unified diagnostic framework. We use the rigorous topological signals from (b) and (c) to expose the visual illusions present in (a).
• Figure 4: Visualizing the Topological Fracture. Comparing latent space interpolations at the two key epochs. The degradation in connectivity (b) is invisible to FID but perfectly captured by the negative Percolation Shift.
• Figure 5: The Topological Signature of GAN Training. We tracked the Percolation Shift $\Delta\varepsilon_c$ during standard GAN training. The curve rises from initialization but stabilizes in a persistent negative regime ($\Delta\varepsilon_c \approx -1.5$, shaded red region). Unlike diffusion models which may initially over-expand and then shrink, this confirms that GANs suffer from inherent Manifold Shrinkage, providing rigorous topological evidence for the classical "Mode Dropping" phenomenon.

Theorems & Definitions (17)

• Definition 1: Random Geometric Graph
• Proposition 1: Finite-Size Scaling Law
• proof
• Proposition 2: Generalization to Non-Uniform Density
• proof
• Theorem 1: Manifold Shrinkage Theorem
• proof
• Remark 1: The Ambiguity of Visual Clumping: Collapse vs. Divergence
• Proposition 3: Sorted Distance Matching Expands the Support
• proof : Proof sketch