Optimizing Federated Graph Learning with Inherent Structural Knowledge and Dual-Densely Connected GNNs

TL;DR

This work tackles non-IID graph data challenges in Federated Graph Learning by explicitly incorporating inherent structural knowledge. It introduces FedDense, which combines a data-level structural vector with a Dual-Densely Connected GNN architecture that densely fuses feature and structure maps across layers, while employing narrow layers and selective parameter sharing to reduce communication and computation. Empirical results on 15 datasets across 4 domains show that FedDense achieves superior accuracy and faster convergence than strong baselines like FedStar, GCFL, and FedAvg, with substantially lower resource demands. The approach offers a practically impactful route to efficient, privacy-preserving, cross-domain graph learning in resource-constrained federated environments.

Abstract

Federated Graph Learning (FGL) is an emerging technology that enables clients to collaboratively train powerful Graph Neural Networks (GNNs) in a distributed manner without exposing their private data. Nevertheless, FGL still faces the challenge of the severe non-Independent and Identically Distributed (non-IID) nature of graphs, which possess diverse node and edge structures, especially across varied domains. Thus, exploring the knowledge inherent in these structures becomes significantly crucial. Existing methods, however, either overlook the inherent structural knowledge in graph data or capture it at the cost of significantly increased resource demands (e.g., FLOPs and communication bandwidth), which can be detrimental to distributed paradigms. Inspired by this, we propose FedDense, a novel FGL framework that optimizes the utilization efficiency of inherent structural knowledge. To better acquire knowledge of diverse and underexploited structures, FedDense first explicitly encodes the structural knowledge inherent within graph data itself alongside node features. Besides, FedDense introduces a Dual-Densely Connected (DDC) GNN architecture that exploits the multi-scale (i.e., one-hop to multi-hop) feature and structure insights embedded in the aggregated feature maps at each layer. In addition to the exploitation of inherent structures, we consider resource limitations in FGL, devising exceedingly narrow layers atop the DDC architecture and adopting a selective parameter sharing strategy to reduce resource costs substantially. We conduct extensive experiments using 15 datasets across 4 different domains, demonstrating that FedDense consistently surpasses baselines by a large margin in training performance, while demanding minimal resources.

Figures (4)

• Figure 1: The Jensen-Shannon divergence heatmap compares feature (left) and structure (right) information among six graph datasets from varied domains. Feature information: empirical feature similarity distribution between all linked nodes; Structure information: concatenation of degree divergence and clustering coefficient distributions.
• Figure 2: An overview of the proposed FedDense framework with 3 GNN layers as an example. The left box represents the local training process with structural patterns decoupling and DDC architecture of each client. The right box illustrates the global Selective Federated sharing scheme.
• Figure 3: Test accuracy curves of FedDense and four baselines along the communication rounds under highly heterogeneous conditions.
• Figure 4: Computation efficiency in four non-IID settings, presenting the minimum average accuracy across five random repetitions and the average FLOPs per client per round for each FGL method. $r$ and $k$ denote the output size of the local GNNs in FedDense and other FGL methods, respectively.