Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leiden-Fusion Partitioning Method for Effective Distributed Training of Graph Embeddings

Yuhe Bai, Camelia Constantin, Hubert Naacke

TL;DR

This work introduces a novel partitioning method, named Leiden-Fusion, designed for large-scale training of graphs with minimal communication, that guarantees that, for an initially connected graph, each partition is a densely connected subgraph with no isolated nodes.

Abstract

In the area of large-scale training of graph embeddings, effective training frameworks and partitioning methods are critical for handling large networks. However, they face two major challenges: 1) existing synchronized distributed frameworks require continuous communication to access information from other machines, and 2) the inability of current partitioning methods to ensure that subgraphs remain connected components without isolated nodes, which is essential for effective training of GNNs since training relies on information aggregation from neighboring nodes. To address these issues, we introduce a novel partitioning method, named Leiden-Fusion, designed for large-scale training of graphs with minimal communication. Our method extends the Leiden community detection algorithm with a greedy algorithm that merges the smallest communities with highly connected neighboring communities. Our method guarantees that, for an initially connected graph, each partition is a densely connected subgraph with no isolated nodes. After obtaining the partitions, we train a GNN for each partition independently, and finally integrate all embeddings for node classification tasks, which significantly reduces the need for network communication and enhances the efficiency of distributed graph training. We demonstrate the effectiveness of our method through extensive evaluations on several benchmark datasets, achieving high efficiency while preserving the quality of the graph embeddings for node classification tasks.

Leiden-Fusion Partitioning Method for Effective Distributed Training of Graph Embeddings

TL;DR

This work introduces a novel partitioning method, named Leiden-Fusion, designed for large-scale training of graphs with minimal communication, that guarantees that, for an initially connected graph, each partition is a densely connected subgraph with no isolated nodes.

Abstract

In the area of large-scale training of graph embeddings, effective training frameworks and partitioning methods are critical for handling large networks. However, they face two major challenges: 1) existing synchronized distributed frameworks require continuous communication to access information from other machines, and 2) the inability of current partitioning methods to ensure that subgraphs remain connected components without isolated nodes, which is essential for effective training of GNNs since training relies on information aggregation from neighboring nodes. To address these issues, we introduce a novel partitioning method, named Leiden-Fusion, designed for large-scale training of graphs with minimal communication. Our method extends the Leiden community detection algorithm with a greedy algorithm that merges the smallest communities with highly connected neighboring communities. Our method guarantees that, for an initially connected graph, each partition is a densely connected subgraph with no isolated nodes. After obtaining the partitions, we train a GNN for each partition independently, and finally integrate all embeddings for node classification tasks, which significantly reduces the need for network communication and enhances the efficiency of distributed graph training. We demonstrate the effectiveness of our method through extensive evaluations on several benchmark datasets, achieving high efficiency while preserving the quality of the graph embeddings for node classification tasks.
Paper Structure (18 sections, 4 equations, 8 figures, 5 tables, 2 algorithms)

This paper contains 18 sections, 4 equations, 8 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background on Graph Embeddings
  3. Related Work
  4. Partitioning Methods
  5. Distributed Training Frameworks
  6. Leiden-Fusion Method
  7. Essential Features for Graph Partitioning
  8. Leiden Community Detection
  9. Community Fusion
  10. Partition Visualization on Karate Dataset
  11. Advantages of the Proposed Two-Step Method:
  12. Experimental Results
  13. Analysis of Partitions
  14. Quality Comparison
  15. Speed Analysis
  16. ...and 3 more sections

Figures (8)

  • Figure 1: Aggregation of nodes A and B in GNNs with different partitioning strategies
  • Figure 2: Visualization of Leiden community detection and fusion process
  • Figure 3: Comparison of partitioning methods on Karate dataset. $\bullet$ Partition 0 $\bullet$ Partition 1
  • Figure 3: Partitioning time comparison on Arxiv dataset across different methods and partitioning numbers.
  • Figure 4: Comparison of subgraph quality on Arxiv dataset
  • ...and 3 more figures

Theorems & Definitions (3)

  • definition thmcounterdefinition: Leiden communities
  • definition thmcounterdefinition: Edge cut
  • definition thmcounterdefinition: Neighbor communities