Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Continuous Geometry-Aware Graph Diffusion via Hyperbolic Neural PDE

Jiaxu Liu, Xinping Yi, Sihao Wu, Xiangyu Yin, Tianle Zhang, Xiaowei Huang, Shi Jin

TL;DR

This work addresses the depth scalability challenge of Hyperbolic Graph Neural Networks by reframing depth as a continuous-time diffusion on hyperbolic space and decoupling layer components through Hyperbolic Neural PDE (HPDE). It introduces HGDE, a hyperbolic graph diffusion equation whose vector flow is guided by learnable diffusivity that captures local and high-order relations, while allowing curvature to diffuse continuously via a learnable $\kappa_t$. The method employs hyperbolic numerical solvers (HEuler, HRK, HAM) and curved-space interpolation to propagate embeddings, and uses a hyperbolic residual mechanism to prevent over-smoothing, achieving superior performance on hierarchical graphs and competitive results on non-hierarchical data. Empirical results across node classification, link prediction, and image-text tasks demonstrate improved accuracy and efficiency, with ablations validating the roles of diffusivity design and residual diffusion. The work highlights the potential of PDE-based non-Euclidean models for scalable, expressive graph representation learning in hyperbolic geometry.

Abstract

While Hyperbolic Graph Neural Network (HGNN) has recently emerged as a powerful tool dealing with hierarchical graph data, the limitations of scalability and efficiency hinder itself from generalizing to deep models. In this paper, by envisioning depth as a continuous-time embedding evolution, we decouple the HGNN and reframe the information propagation as a partial differential equation, letting node-wise attention undertake the role of diffusivity within the Hyperbolic Neural PDE (HPDE). By introducing theoretical principles \textit{e.g.,} field and flow, gradient, divergence, and diffusivity on a non-Euclidean manifold for HPDE integration, we discuss both implicit and explicit discretization schemes to formulate numerical HPDE solvers. Further, we propose the Hyperbolic Graph Diffusion Equation (HGDE) -- a flexible vector flow function that can be integrated to obtain expressive hyperbolic node embeddings. By analyzing potential energy decay of embeddings, we demonstrate that HGDE is capable of modeling both low- and high-order proximity with the benefit of local-global diffusivity functions. Experiments on node classification and link prediction and image-text classification tasks verify the superiority of the proposed method, which consistently outperforms various competitive models by a significant margin.

Continuous Geometry-Aware Graph Diffusion via Hyperbolic Neural PDE

TL;DR

This work addresses the depth scalability challenge of Hyperbolic Graph Neural Networks by reframing depth as a continuous-time diffusion on hyperbolic space and decoupling layer components through Hyperbolic Neural PDE (HPDE). It introduces HGDE, a hyperbolic graph diffusion equation whose vector flow is guided by learnable diffusivity that captures local and high-order relations, while allowing curvature to diffuse continuously via a learnable . The method employs hyperbolic numerical solvers (HEuler, HRK, HAM) and curved-space interpolation to propagate embeddings, and uses a hyperbolic residual mechanism to prevent over-smoothing, achieving superior performance on hierarchical graphs and competitive results on non-hierarchical data. Empirical results across node classification, link prediction, and image-text tasks demonstrate improved accuracy and efficiency, with ablations validating the roles of diffusivity design and residual diffusion. The work highlights the potential of PDE-based non-Euclidean models for scalable, expressive graph representation learning in hyperbolic geometry.

Abstract

While Hyperbolic Graph Neural Network (HGNN) has recently emerged as a powerful tool dealing with hierarchical graph data, the limitations of scalability and efficiency hinder itself from generalizing to deep models. In this paper, by envisioning depth as a continuous-time embedding evolution, we decouple the HGNN and reframe the information propagation as a partial differential equation, letting node-wise attention undertake the role of diffusivity within the Hyperbolic Neural PDE (HPDE). By introducing theoretical principles \textit{e.g.,} field and flow, gradient, divergence, and diffusivity on a non-Euclidean manifold for HPDE integration, we discuss both implicit and explicit discretization schemes to formulate numerical HPDE solvers. Further, we propose the Hyperbolic Graph Diffusion Equation (HGDE) -- a flexible vector flow function that can be integrated to obtain expressive hyperbolic node embeddings. By analyzing potential energy decay of embeddings, we demonstrate that HGDE is capable of modeling both low- and high-order proximity with the benefit of local-global diffusivity functions. Experiments on node classification and link prediction and image-text classification tasks verify the superiority of the proposed method, which consistently outperforms various competitive models by a significant margin.
Paper Structure (27 sections, 1 theorem, 44 equations, 5 figures, 8 tables, 3 algorithms)

This paper contains 27 sections, 1 theorem, 44 equations, 5 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Riemannian Geometry and Hyperbolic Space
  4. Diffusion Equations
  5. Graph Diffusion Equation
  6. Hyperbolic Numerical Integrators
  7. Hyperbolic Projective Explicit Scheme
  8. H-Explicit Euler (HEuler)
  9. H-Runge-Kutta (HRK)
  10. Hyperbolic Projective Implicit Scheme
  11. H-Implicit Adams–Moulton (HAM)
  12. Interpolation on Curved Space
  13. Diffusing Graphs in Hyperbolic Space
  14. Hyperbolic Graph Diffusion Equation
  15. Gradient
  16. ...and 12 more sections

Key Result

proposition thmcounterproposition

For any step size $0<\delta<\tau$, the interpolation $\mathbf{h}(t+\delta)$ via Eq. (eq:hyperbolic-linear-interpolation) is on the geodesic between $\mathbf{h}(t)$ and $\mathbf{h}(t+\tau)$ on the manifold, and $\frac{d_\mathbb{D}^\kappa(\mathbf{h}(t), \mathbf{h}(t+\delta))}{ d_\mathbb{D}^\kappa(\mat

Figures (5)

  • Figure 1: (a-c) Illustration of various numerical integration methods with comparison to RGD. In each time-step, an explicit scheme calibrates the vector field within only the tangent space of time $t$, while an implicit scheme requires multiple tangent spaces to estimate future slopes, thus requiring parallel transport for aligning the directions of vectors in different spaces. (d) Illustration of hyperbolic interpolation method.
  • Figure 2: Schematic of HGDE. (a) The pipeline of our method includes hyperbolic projection, feature transformation, and HPDE block that integrates the GDE. After that, a decoder is applied to the embeddings for specific downstream tasks. (b) The visualization of the diffusion process within the HPDE block: first, map local gradients of $\mathbf{z}_i$ onto the tangent space, calculate the diffusivity, and diverge to obtain the vector flow (green arrow), then perform one-step integration on the manifold with the guidance of continuous curvature diffusion. (c) The details of attention-powered local-global diffusivity function.
  • Figure 3: Hyperbolic Dirichlet energy $f^{\kappa}_\mathrm{DE}(\cdot)$ variation through $t$ on Cora (left) and CiteSeer (right). Models are compared with different integrators w or w/o hyperbolic residual.
  • Figure 4: Averaged node classification performance comparison of models with different diffusivity functions on various datasets.
  • Figure 5: Cora diffusivity (400 node sampled from $\mathbb{D}^2_{\kappa}$ embeddings) produced by $a^{\mathrm{ldiff}}$ (left) and $a^{\mathrm{lgdiff}}$ (right), blue and red lines denote local and global attention; bolder lines indicate more attentiveness.

Theorems & Definitions (8)

  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • proposition thmcounterproposition: proved in Appendix D
  • definition thmcounterdefinition
  • definition thmcounterdefinition: Probability Measure
  • definition thmcounterdefinition: Wasserstein Distance
  • remark thmcounterremark
  • proof