Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement
Yinlin Zhu, Xunkai Li, Jishuo Jia, Miao Hu, Di Wu, Meikang Qiu
TL;DR
This work investigates training graph foundation models in a federated, cross-domain, cross-task setting and identifies knowledge entanglement as a key barrier in naive federated pre-training. It introduces FedGFM+, a dual-module framework with AncDAI for global domain-aware initialization and AdaDPP for local domain-sensitive prompts, to disentangle domain knowledge and enable adaptive downstream performance. Empirical results across eight heterogeneous benchmarks show FedGFM+ consistently outperforming isolated, FL/FGL, and federated centralized-GFM baselines, with ablations confirming the necessity of both modules. The approach demonstrates that combining federated learning with graph foundation modeling can unlock cross-silo collaboration while maintaining generalization across domains and tasks, albeit with privacy considerations that warrant future formal guarantees.
Abstract
Recent advances in graph machine learning have shifted to data-centric paradigms, driven by two emerging fields: (1) Federated graph learning (FGL) enables multi-client collaboration but faces challenges from data and task heterogeneity, limiting its practicality; (2) Graph foundation models (GFM) offer strong domain generalization but are usually trained on single machines, missing out on cross-silo data and resources. These paradigms are complementary, and their integration brings notable benefits. Motivated by this, we propose FedGFM, a novel decentralized GFM training paradigm. However, a key challenge is knowledge entanglement, where multi-domain knowledge merges into indistinguishable representations, hindering downstream adaptation. To address this, we present FedGFM+, an enhanced framework with two core modules to reduce knowledge entanglement: (1) AncDAI: A global anchor-based domain-aware initialization strategy. Before pre-training, each client encodes its local graph into domain-specific prototypes that serve as semantic anchors. Synthetic embeddings around these anchors initialize the global model. We theoretically prove these prototypes are distinguishable across domains, providing a strong inductive bias to disentangle domain-specific knowledge. (2) AdaDPP: A local adaptive domain-sensitive prompt pool. Each client learns a lightweight graph prompt capturing domain semantics during pre-training. During fine-tuning, prompts from all clients form a pool from which the GFM selects relevant prompts to augment target graph attributes, improving downstream adaptation. FedGFM+ is evaluated on 8 diverse benchmarks across multiple domains and tasks, outperforming 20 baselines from supervised learning, FGL, and federated GFM variants.