FedSSP: Federated Graph Learning with Spectral Knowledge and Personalized Preference

TL;DR

This work innovatively reveals that the spectral nature of graphs can well reflect inherent domain structural shifts and proposes the pFGL framework FedSSP which Shares generic Spectral knowledge while satisfying graph Preferences.

Abstract

Personalized Federated Graph Learning (pFGL) facilitates the decentralized training of Graph Neural Networks (GNNs) without compromising privacy while accommodating personalized requirements for non-IID participants. In cross-domain scenarios, structural heterogeneity poses significant challenges for pFGL. Nevertheless, previous pFGL methods incorrectly share non-generic knowledge globally and fail to tailor personalized solutions locally under domain structural shift. We innovatively reveal that the spectral nature of graphs can well reflect inherent domain structural shifts. Correspondingly, our method overcomes it by sharing generic spectral knowledge. Moreover, we indicate the biased message-passing schemes for graph structures and propose the personalized preference module. Combining both strategies, we propose our pFGL framework FedSSP which Shares generic Spectral knowledge while satisfying graph Preferences. Furthermore, We perform extensive experiments on cross-dataset and cross-domain settings to demonstrate the superiority of our framework. The code is available at https://github.com/OakleyTan/FedSSP.

Figures (4)

• Figure 1: Problem illustration. We illustrate the challenges of the cross-domain scenario. (a) Considering the domain structural shifts, clients struggle with knowledge conflict caused by non-generic sharing which arises from the shifts, thus leading to unpromising global collaboration. (b) The aggregated message-passing scheme suffers from inconsistent preferences that remain unsatisfied of specific datasets in this scenario. Consequently, it leads to unsuitable features of graphs in local applications. (c) The heat map of Jensen-Shannon divergence of algebraic connectivity 1973_algebraic and eigenvalues distributions among six datasets from three different domains. Spectral characteristics exhibit significant biases across domains but are more similar within a same domain.
• Figure 2: Architecture illustration of FedSSP. The left box (a) refers to Generic Spectral Knowledge Sharing (GSKS), where we address knowledge conflict and promote effective global collaboration by sharinggeneric spectral knowledge extracted from spectral encoders $\phi^{e}$ and $\phi^{f}$ while retainingnon-generic in other components. The right box (b) represents Personalized Graph Preference Adjustment (PGPA), where we leverage preference module guided by $\mathcal{L}_{i}^{PGPA}$ for satisfying inconsistent preferences and achieving suitable feature of datasets locally. These two boxes correspondingly refer to the two core strategies of our framework FedSSP.
• Figure 3: Test accuracy curves of FedSSP and six other methods along the communication rounds on our three different settings(SM, SM-CV, SM-SN-CV). The y-axis range is from 65 to 85 for all settings.
• Figure 4: Analysis on hyper-parameter in FedSSP. Graph classification results under different $\tau$, $\mu$, attention heads, and hidden dimensions. Colors green, blue, and yellow refer to performance on single, double, and multi-domain settings (SM, SM-CV, SM-SN-CV). The dashed lines of corresponding colors represent the baseline test accuracy for each setting, which includes only the GSKS strategy.