Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach

TL;DR

FUELS tackles spatio-temporal heterogeneity in federated forecasting by introducing a dual semantic alignment framework that combines a temporal intra-client hard-negative filtering contrastive task with a spatial inter-client prototype-based contrastive task. The server aggregates lightweight periodicity-aware prototypes using a Jensen-Shannon divergence to tailor client-specific global prototypes, enabling effective inter-client knowledge transfer with minimal communication. Empirical results across multiple traffic datasets show FUELS achieves superior forecasting while reducing communication costs by about 94%, and analyses reveal the complementary benefits of both intra- and inter-client losses, as well as the robustness to privacy-preserving noise. The approach offers a practical, efficient pathway to personalized FL for spatio-temporal prediction and can be extended to other spatio-temporal domains and model architectures.

Abstract

The existing federated learning (FL) methods for spatio-temporal forecasting fail to capture the inherent spatio-temporal heterogeneity, which calls for personalized FL (PFL) methods to model the spatio-temporally variant patterns. While contrastive learning approach is promising in addressing spatio-temporal heterogeneity, the existing methods are noneffective in determining negative pairs and can hardly apply to PFL paradigm. To tackle this limitation, we propose a novel PFL method, named Federated dUal sEmantic aLignment-based contraStive learning (FUELS), which can adaptively align positive and negative pairs based on semantic similarity, thereby injecting precise spatio-temporal heterogeneity into the latent representation space by auxiliary contrastive tasks. From temporal perspective, a hard negative filtering module is introduced to dynamically align heterogeneous temporal representations for the supplemented intra-client contrastive task. From spatial perspective, we design lightweight-but-efficient prototypes as client-level semantic representations, based on which the server evaluates spatial similarity and yields client-customized global prototypes for the supplemented inter-client contrastive task. Extensive experiments demonstrate that FUELS outperforms state-of-the-art methods, with communication cost decreasing by around 94%.

Figures (9)

• Figure 1: Spatio-temporal heterogeneity inside traffic flows.
• Figure 2: An overview of the proposed FUELS. (a) Each client performs local training by the supplemented inter- and intra-client contrastive loss items for spatio-temporal heterogeneity. (b) The designed periodicity-aware prototype works as the communication carrier. (c) The JSD-based aggregation generates client-customized global prototypes.
• Figure 3: Comparison of different methods in terms of prediction values and CDFs of MSE.
• Figure 4: Training MSE versus communication amounts on (a) SMS and (b) Net datasets.
• Figure 5: Performance comparison of FUELS and four variants on Net dataset.

Theorems & Definitions (6)

• Theorem 4.2
• Theorem 4.6
• Theorem 3.2
• proof
• Theorem 3.6
• proof