Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularity

Zichen Liu, Wei Zhang, Tiejun Li

TL;DR

This work shows that naive diffusion in ambient space suffers a multiscale singularity when data are constrained to a manifold, with the normal score component blowing up as $\sigma\to0$. By decomposing the perturbed score into tangential and normal parts, the authors derive theoretical insights and two practical remedies: Niso-DM, which injects non-isotropic normal noise to balance scales, and Tango-DM, which trains only the tangential score. They validate these methods on diverse manifolds (hyperplanes, meshes, high-dimensional groups, and biomolecular configurations), reporting improved alignment with target distributions and reduced tangential-score error. The proposed approaches offer a scalable, geometry-aware path to accurate diffusion-based generation on known manifolds without requiring explicit geodesic or heat-kernel computations. Overall, the work advances diffusion modeling for manifold-structured data by addressing intrinsic score singularities and proposing robust, geometry-friendly training and sampling strategies.

Abstract

Euclidean diffusion models have achieved remarkable success in generative modeling across diverse domains, and they have been extended to manifold cases in recent advances. Instead of explicitly utilizing the structure of special manifolds as studied in previous works, in this paper we investigate direct sampling of the Euclidean diffusion models for general manifold-structured data. We reveal the multiscale singularity of the score function in the ambient space, which hinders the accuracy of diffusion-generated samples. We then present an elaborate theoretical analysis of the singularity structure of the score function by decomposing it along the tangential and normal directions of the manifold. To mitigate the singularity and improve the sampling accuracy, we propose two novel methods: (1) Niso-DM, which reduces the scale discrepancies in the score function by utilizing a non-isotropic noise, and (2) Tango-DM, which trains only the tangential component of the score function using a tangential-only loss function. Numerical experiments demonstrate that our methods achieve superior performance on distributions over various manifolds with complex geometries.

Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularity

TL;DR

This work shows that naive diffusion in ambient space suffers a multiscale singularity when data are constrained to a manifold, with the normal score component blowing up as . By decomposing the perturbed score into tangential and normal parts, the authors derive theoretical insights and two practical remedies: Niso-DM, which injects non-isotropic normal noise to balance scales, and Tango-DM, which trains only the tangential score. They validate these methods on diverse manifolds (hyperplanes, meshes, high-dimensional groups, and biomolecular configurations), reporting improved alignment with target distributions and reduced tangential-score error. The proposed approaches offer a scalable, geometry-aware path to accurate diffusion-based generation on known manifolds without requiring explicit geodesic or heat-kernel computations. Overall, the work advances diffusion modeling for manifold-structured data by addressing intrinsic score singularities and proposing robust, geometry-friendly training and sampling strategies.

Abstract

Euclidean diffusion models have achieved remarkable success in generative modeling across diverse domains, and they have been extended to manifold cases in recent advances. Instead of explicitly utilizing the structure of special manifolds as studied in previous works, in this paper we investigate direct sampling of the Euclidean diffusion models for general manifold-structured data. We reveal the multiscale singularity of the score function in the ambient space, which hinders the accuracy of diffusion-generated samples. We then present an elaborate theoretical analysis of the singularity structure of the score function by decomposing it along the tangential and normal directions of the manifold. To mitigate the singularity and improve the sampling accuracy, we propose two novel methods: (1) Niso-DM, which reduces the scale discrepancies in the score function by utilizing a non-isotropic noise, and (2) Tango-DM, which trains only the tangential component of the score function using a tangential-only loss function. Numerical experiments demonstrate that our methods achieve superior performance on distributions over various manifolds with complex geometries.
Paper Structure (46 sections, 2 theorems, 64 equations, 3 figures, 9 tables, 3 algorithms)

This paper contains 46 sections, 2 theorems, 64 equations, 3 figures, 9 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Backgrounds and preliminaries
  3. Diffusion models
  4. Riemannian manifolds and notation
  5. Scale discrepancy of score functions
  6. Methods
  7. Niso-DM: Perturb data with non-isotropic noise
  8. Tango-DM: Learn only the tangential component of the score function
  9. Generation
  10. Reverse SDE
  11. Annealing SDE on manifolds
  12. Experiments
  13. The hyperplane in 3D space
  14. Mesh data
  15. High-dimensional special orthogonal group
  16. ...and 31 more sections

Key Result

Theorem 3.1

Let $P(x)\in \mathbb{R}^{n\times n}$ denote the projection matrix at $x\in \mathcal{M}$. Assume that $\int_{\mathcal{M}} \|x\| p_0(x) \mathrm{d}\sigma_{\mathcal{M}}(x) <+\infty$ and $M_1 = \sup_{x\in \mathcal{M}}\max_{1\le i,j,j' \le n}\left|\frac{\partial P_{ij}}{\partial x_{j'}}(x)\right|<+\infty$

Figures (3)

  • Figure 1: The average error of the tangential component of the learned score function in the $x-y$ plane, along the $z$-axis and $t$-axis, with $\sigma_{\min}=0.01$. From top to bottom, the plots correspond to the vanilla algorithm (Iso-DM), our proposed Niso-DM and Tango-DM, all using the rescaling technique.
  • Figure 2: Illustration of the Alanine dipeptide system: The dihedral angles $\phi$ and $\psi$ are defined by atoms whose indices are $5,7,9, 15$ and $7,9, 15, 17$, respectively. This figure is from liu2025rddpm.
  • Figure 3: Ablation Studies for $\mathrm{SO}(10)$: The solid line denotes the mean, while the shaded area indicates the standard deviation. (a) The error of the distribution generated by the Reverse SDE algorithm under the Iso-DM (red) and Niso-DM (blue) methods with varying $\sigma_{\min}$. (b) The impact of $c_{\mathrm{niso}}$ on the error with $\sigma_{\min}=0.001$. (c) The impact of $c_{\mathrm{tango}}$ on the error with $\sigma_{\min}=0.001$.

Theorems & Definitions (2)

  • Theorem 3.1
  • Theorem 4.1