Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Experts

Zihao Guo, Qingyun Sun, Haonan Yuan, Xingcheng Fu, Min Zhou, Yisen Gao, Jianxin Li

TL;DR

This paper proposes a novel Graph Mixture of Riemannian Experts (GraphMoRE) framework to effectively tackle topological heterogeneity by personalized fine-grained topology geometry pattern preservation, and proposes a topology-aware gating mechanism to select the optimal embedding space for each node.

Abstract

Real-world graphs have inherently complex and diverse topological patterns, known as topological heterogeneity. Most existing works learn graph representation in a single constant curvature space that is insufficient to match the complex geometric shapes, resulting in low-quality embeddings with high distortion. This also constitutes a critical challenge for graph foundation models, which are expected to uniformly handle a wide variety of diverse graph data. Recent studies have indicated that product manifold gains the possibility to address topological heterogeneity. However, the product manifold is still homogeneous, which is inadequate and inflexible for representing the mixed heterogeneous topology. In this paper, we propose a novel Graph Mixture of Riemannian Experts (GraphMoRE) framework to effectively tackle topological heterogeneity by personalized fine-grained topology geometry pattern preservation. Specifically, to minimize the embedding distortion, we propose a topology-aware gating mechanism to select the optimal embedding space for each node. By fusing the outputs of diverse Riemannian experts with learned gating weights, we construct personalized mixed curvature spaces for nodes, effectively embedding the graph into a heterogeneous manifold with varying curvatures at different points. Furthermore, to fairly measure pairwise distances between different embedding spaces, we present a concise and effective alignment strategy. Extensive experiments on real-world and synthetic datasets demonstrate that our method achieves superior performance with lower distortion, highlighting its potential for modeling complex graphs with topological heterogeneity, and providing a novel architectural perspective for graph foundation models.

GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Experts

TL;DR

This paper proposes a novel Graph Mixture of Riemannian Experts (GraphMoRE) framework to effectively tackle topological heterogeneity by personalized fine-grained topology geometry pattern preservation, and proposes a topology-aware gating mechanism to select the optimal embedding space for each node.

Abstract

Real-world graphs have inherently complex and diverse topological patterns, known as topological heterogeneity. Most existing works learn graph representation in a single constant curvature space that is insufficient to match the complex geometric shapes, resulting in low-quality embeddings with high distortion. This also constitutes a critical challenge for graph foundation models, which are expected to uniformly handle a wide variety of diverse graph data. Recent studies have indicated that product manifold gains the possibility to address topological heterogeneity. However, the product manifold is still homogeneous, which is inadequate and inflexible for representing the mixed heterogeneous topology. In this paper, we propose a novel Graph Mixture of Riemannian Experts (GraphMoRE) framework to effectively tackle topological heterogeneity by personalized fine-grained topology geometry pattern preservation. Specifically, to minimize the embedding distortion, we propose a topology-aware gating mechanism to select the optimal embedding space for each node. By fusing the outputs of diverse Riemannian experts with learned gating weights, we construct personalized mixed curvature spaces for nodes, effectively embedding the graph into a heterogeneous manifold with varying curvatures at different points. Furthermore, to fairly measure pairwise distances between different embedding spaces, we present a concise and effective alignment strategy. Extensive experiments on real-world and synthetic datasets demonstrate that our method achieves superior performance with lower distortion, highlighting its potential for modeling complex graphs with topological heterogeneity, and providing a novel architectural perspective for graph foundation models.

Paper Structure

This paper contains 43 sections, 11 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Riemannian Graph Learning
  4. Graph Mixture of Experts
  5. Preliminary
  6. Riemannian Manifold
  7. Manifold.
  8. Constant Curvature Spaces.
  9. Problem Formulation
  10. Notations.
  11. Problem Definition.
  12. GraphMoRE
  13. Diverse Riemannian Experts
  14. Riemannian GNN.
  15. Diversity of Experts.
  16. ...and 28 more sections

Figures (4)

  • Figure 1: The overall framework of GraphMoRE. We first design diverse Rimannian experts in terms of both the type of curvature spaces and the degree of curving in space. Then we estimate the geometric properties around nodes from the perspective of local topology characterization, embed nodes into personalized mixed curvature spaces by fusing diverse Riemannian experts with the goal of minimizing embedding distortion. Finally, we measure pairwise distances through alignment strategy.
  • Figure 2: Performance comparison on synthetic graphs with topological heterogeneity evaluated by AUC (%).
  • Figure 3: Visualization of distortion of different local topology structures in Cora. Closer to red, more distortion.
  • Figure A.1: Statistics of sectional curvature distribution.