Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsification

TL;DR

This work addresses the high bidirectional communication cost in Federated Knowledge Graph Embedding (FKGE) by demonstrating that universal per-entity embedding precision reduction harms convergence and proposing FedS, a modular method that reduces transmitted parameter size via Entity-Wise Top-K Sparsification. FedS comprises upstream Top-K sparsification, downstream personalized Top-K sparsification, and an Intermittent Synchronization mechanism to handle FKGE heterogeneity. Empirical results on three FB15k-237-based datasets show substantial reductions in communication while preserving near-FedEP performance, outperforming simple embedding-dimension reductions and highlighting the method's practicality for bandwidth-constrained FKGE deployments. The approach is compatible with multiple FKGE backbones and offers a configurable trade-off between communication efficiency and convergence speed, with concrete guidance on synchronization intervals and sparsification ratios. Overall, FedS provides a principled, scalable path to deploy FKGE in federated, resource-limited environments.

Abstract

Federated Knowledge Graphs Embedding learning (FKGE) encounters challenges in communication efficiency stemming from the considerable size of parameters and extensive communication rounds. However, existing FKGE methods only focus on reducing communication rounds by conducting multiple rounds of local training in each communication round, and ignore reducing the size of parameters transmitted within each communication round. To tackle the problem, we first find that universal reduction in embedding precision across all entities during compression can significantly impede convergence speed, underscoring the importance of maintaining embedding precision. We then propose bidirectional communication-efficient FedS based on Entity-Wise Top-K Sparsification strategy. During upload, clients dynamically identify and upload only the Top-K entity embeddings with the greater changes to the server. During download, the server first performs personalized embedding aggregation for each client. It then identifies and transmits the Top-K aggregated embeddings to each client. Besides, an Intermittent Synchronization Mechanism is used by FedS to mitigate negative effect of embedding inconsistency among shared entities of clients caused by heterogeneity of Federated Knowledge Graph. Extensive experiments across three datasets showcase that FedS significantly enhances communication efficiency with negligible (even no) performance degradation.

Figures (3)

• Figure 1: Overall procedure of FedS at each communication round.
• Figure 2: FB15k-237-R3
• Figure 3: FB15k-237-R5