Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random Walk Guided Hyperbolic Graph Distillation

Yunbo Long, Liming Xu, Stefan Schoepf, Alexandra Brintrup

TL;DR

HyDRO tackles Euclidean graph distillation limitations by distilling graphs in hyperbolic space to better capture hierarchical structure and dynamic information. It embeds node features in the Poincaré ball $\mathbb{H}^d$ and optimizes a spectral-gap objective $g$ to align random-walk properties between original and condensed graphs. The approach yields state-of-the-art or competitive results on node classification and link prediction, while enhancing continual graph learning, neural architecture search transferability, privacy, and denoising robustness. This work provides a scalable hyperbolic condensation framework with broad impact on downstream graph tasks and privacy-aware learning.

Abstract

Graph distillation (GD) is an effective approach to extract useful information from large-scale network structures. However, existing methods, which operate in Euclidean space to generate condensed graphs, struggle to capture the inherent tree-like geometry of real-world networks, resulting in distilled graphs with limited task-specific information for downstream tasks. Furthermore, these methods often fail to extract dynamic properties from graphs, which are crucial for understanding information flow and facilitating graph continual learning. This paper presents the Hyperbolic Graph Distillation with Random Walks Optimization (HyDRO), a novel graph distillation approach that leverages hyperbolic embeddings to capture complex geometric patterns and optimize the spectral gap in hyperbolic space. Experiments show that HyDRO demonstrates strong task generalization, consistently outperforming state-of-the-art methods in both node classification and link prediction tasks. HyDRO also effectively preserves graph random walk properties, producing condensed graphs that achieve enhanced performance in continual graph learning. Additionally, HyDRO achieves competitive results on mainstream graph distillation benchmarks, while maintaining a strong balance between privacy and utility, and exhibiting robust resistance to noises.

Random Walk Guided Hyperbolic Graph Distillation

TL;DR

HyDRO tackles Euclidean graph distillation limitations by distilling graphs in hyperbolic space to better capture hierarchical structure and dynamic information. It embeds node features in the Poincaré ball and optimizes a spectral-gap objective to align random-walk properties between original and condensed graphs. The approach yields state-of-the-art or competitive results on node classification and link prediction, while enhancing continual graph learning, neural architecture search transferability, privacy, and denoising robustness. This work provides a scalable hyperbolic condensation framework with broad impact on downstream graph tasks and privacy-aware learning.

Abstract

Graph distillation (GD) is an effective approach to extract useful information from large-scale network structures. However, existing methods, which operate in Euclidean space to generate condensed graphs, struggle to capture the inherent tree-like geometry of real-world networks, resulting in distilled graphs with limited task-specific information for downstream tasks. Furthermore, these methods often fail to extract dynamic properties from graphs, which are crucial for understanding information flow and facilitating graph continual learning. This paper presents the Hyperbolic Graph Distillation with Random Walks Optimization (HyDRO), a novel graph distillation approach that leverages hyperbolic embeddings to capture complex geometric patterns and optimize the spectral gap in hyperbolic space. Experiments show that HyDRO demonstrates strong task generalization, consistently outperforming state-of-the-art methods in both node classification and link prediction tasks. HyDRO also effectively preserves graph random walk properties, producing condensed graphs that achieve enhanced performance in continual graph learning. Additionally, HyDRO achieves competitive results on mainstream graph distillation benchmarks, while maintaining a strong balance between privacy and utility, and exhibiting robust resistance to noises.
Paper Structure (46 sections, 30 equations, 10 figures, 8 tables)

This paper contains 46 sections, 30 equations, 10 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Poincaré Ball Manifold
  4. Lazy Random Walk Matrix
  5. Spectral Gap
  6. HyDRO
  7. Hyperbolic Graph Distillation
  8. Spectral Gap Alignment
  9. Pipeline
  10. Experimental Settings
  11. Datasets.
  12. Baselines.
  13. Evaluation Settings.
  14. Setup and Hyperparameters.
  15. Results and Discussion
  16. ...and 31 more sections

Figures (10)

  • Figure 1: The pipeline of the proposed approach---HyDRO---for graph distillation.
  • Figure 2: Performance of graph distillation methods on mainstream datasets under the graph continual learning setting.
  • Figure 3: Commute time between pairs of nodes on Citeseer dataset (reduction rate 3.6%). ( Note: x and y-axis denote the node ID. The values closer to the original graph (\ref{['fig:ct_original']}) indicate better preservation of random walk properties. The color bars denote the value of commute time.)
  • Figure 4: Commute time between pairs of nodes on Citeseer dataset (reduction rate 0.90%). ( Note: x and y-axis denote the node ID. The values closer to the original graph (\ref{['fig:ct_original']}) indicate better preservation of random walk properties. The color bars denote the value of commute time.)
  • Figure 5: Commute time between pairs of nodes on Citeseer dataset (reduction rate 1.8%). ( Note: x and y-axis denote the node ID. The values closer to the original graph (\ref{['fig:ct_original']}) indicate better preservation of random walk properties. The color bars denote the value of commute time.)
  • ...and 5 more figures