FedSheafHN: Personalized Federated Learning on Graph-structured Data

TL;DR

The paper addresses heterogeneity in graph-structured data within Federated Learning by introducing FedSheafHN, which builds a server-side collaboration graph from client subgraphs, applies neural sheaf diffusion to refine inter-client representations, and uses an attention-enhanced hypernetwork to generate highly personalized client models in parallel. The approach jointly learns the diffusion-based client representations and the hypernetwork parameters, optimizing the overall objective $\arg\min_{\varphi, \theta, \Omega} \mathcal{L}(\Omega)$. Empirical results across six graph-structured datasets show FedSheafHN outperforms baselines, converges rapidly, and generalizes to new clients with minimal overhead. This framework offers a scalable, privacy-aware path to personalized GNNs in federated settings and has potential impact on social networks, recommender systems, and patient networks where data heterogeneity is prevalent.

Abstract

Personalized subgraph Federated Learning (FL) is a task that customizes Graph Neural Networks (GNNs) to individual client needs, accommodating diverse data distributions. However, applying hypernetworks in FL, while aiming to facilitate model personalization, often encounters challenges due to inadequate representation of client-specific characteristics. To overcome these limitations, we propose a model called FedSheafHN, using enhanced collaboration graph embedding and efficient personalized model parameter generation. Specifically, our model embeds each client's local subgraph into a server-constructed collaboration graph. We utilize sheaf diffusion in the collaboration graph to learn client representations. Our model improves the integration and interpretation of complex client characteristics. Furthermore, our model ensures the generation of personalized models through advanced hypernetworks optimized for parallel operations across clients. Empirical evaluations demonstrate that FedSheafHN outperforms existing methods in most scenarios, in terms of client model performance on various graph-structured datasets. It also has fast model convergence and effective new clients generalization.

Figures (5)

• Figure 1: The framework of FedSheafHN.
• Figure 2: Convergence plots for the non-overlapping scenario with 20 clients.
• Figure 3: Convergence plots for the overlapping scenario with 30 clients.
• Figure 4: Results of generalization to new clients on the ogbn-arxiv dataset. "Train" represents the average test accuracy for trained clients, while "New" indicates the average test accuracy for newly joined clients.
• Figure 5: Results on Computer by varying local epochs in different scenarios.