STHFL: Spatio-Temporal Heterogeneous Federated Learning

TL;DR

The paper tackles federated learning under spatio-temporal heterogeneity, where client data distributions are non-iid across clients and each client experiences sequential tasks, causing intra-domain drift and forgetting. It introduces GLDP, a framework that combines federated representation learning with local/global prototype learning, updating prototypes via moving averages and aligning local prototypes with global ones to counter long-tailed class distributions while protecting privacy. Empirical results on CIFAR-10/100 and TinyImageNet show that GLDP achieves superior global and local accuracy compared with baselines, demonstrating strong generalization across domains and robustness to continual task sequences. The approach offers a practical path for real-world FL deployments with nonstationary, unbalanced data.

Abstract

Federated learning is a new framework that protects data privacy and allows multiple devices to cooperate in training machine learning models. Previous studies have proposed multiple approaches to eliminate the challenges posed by non-iid data and inter-domain heterogeneity issues. However, they ignore the \textbf{spatio-temporal} heterogeneity formed by different data distributions of increasing task data in the intra-domain. Moreover, the global data is generally a long-tailed distribution rather than assuming the global data is balanced in practical applications. To tackle the \textbf{spatio-temporal} dilemma, we propose a novel setting named \textbf{Spatio-Temporal Heterogeneity} Federated Learning (STHFL). Specially, the Global-Local Dynamic Prototype (GLDP) framework is designed for STHFL. In GLDP, the model in each client contains personalized layers which can dynamically adapt to different data distributions. For long-tailed data distribution, global prototypes are served as complementary knowledge for the training on classes with few samples in clients without leaking privacy. As tasks increase in clients, the knowledge of local prototypes generated in previous tasks guides for training in the current task to solve catastrophic forgetting. Meanwhile, the global-local prototypes are updated through the moving average method after training local prototypes in clients. Finally, we evaluate the effectiveness of GLDP, which achieves remarkable results compared to state-of-the-art methods in STHFL scenarios.

Figures (4)

• Figure 1: The intuitive description of spatio-temporal heterogeneous federated learning. (a) Spatial heterogeneity: The local distribution across clients is non-iid, and the class distribution is long-tail. (b) Temporal heterogeneity: The data inside the client is continuously increasing.
• Figure 2: Illustration of GLDP. Simplified schematization of our method that solves spatio-temporal heterogeneous federated learning problem via Federated representation learning and Federated prototype learning. (1) Federated represention learning. The local client trains the personalization model to get the shared representation layer ($\mu$) and the personalization layer to fit the non-iid distribution. (2) Federated prototype learning.$\mathcal{L}_{\mathrm{LP}}$: Guide the learning of post-stage $\mathbf{C}_{i}^{n,(m)}$ based on preserved local prototype knowledge $\hat{\mathbf{C}}_{i}^{n,(m)}$ to reduce local model catastrophic forgetting inherent to temporal heterogeneity. $\mathcal{L}_{\mathrm{GP}}$: By making the local prototype approximate the global prototype to learn global knowledge, reduce the global-local model bias generated by long-tail. Compute the global model parameter $\Theta ^{k,(m)}$ and update the global prototype set $\{\widetilde{\mathbf{C}}^{n}\}^z_{n=1}$ based on uploaded $\{\mathbf{C}^{n,(m)}_{i}\}_{i=1}^{N_k}$ on the central server.
• Figure 3: The $\mathcal{A}^{glo}$ (%) of different FL methods on CIFAR10 and CIFAR100 with different IFs. (a) [100, 4, 1]; (b) [100, 5, 1]; (c) [100, 20, 1]; (d) [100, 30, 1].
• Figure 4: Analysis of different stage tasks at client participants on CIFAR10 and CIFAR100 with $\mathcal{A}^{sel}$. CIFAR10: IF = 10: (a) [20, 4, 5]; (b) [20, 5, 5]; IF = 50: (c) [20, 4, 5]; (d) [20, 5, 5]; IF = 100: (e) [20, 4, 5]; (f) [20, 5, 5]; CIFAR100: IF = 10: (g) [20, 20, 5]; (h) [20, 30, 5]; IF = 50: (i) [20, 20, 5]; (j) [20, 30, 5]; IF = 100: (k) [20, 20, 5]; (l) [20, 30, 5];