FedMeS: Personalized Federated Continual Learning Leveraging Local Memory

TL;DR

FedMeS tackles Personalized Federated Continual Learning under non-IID task streams by equipping each client with a small local memory used both to calibrate gradient updates during training and to enable KNN-based Gaussian inference during testing. The framework blends local memory-driven gradient corrections with a dynamic regularization that links local and global knowledge, and employs a task-oblivious inference pipeline that combines a personalized model with memory-based predictions. Theoretical convergence results under standard SGD assumptions show the memory-guided updates drive local losses toward optima, while experiments across diverse PFCL benchmarks demonstrate consistent improvements in average accuracy and forgetting rates compared to strong baselines, including FedWeIT. FedMeS also demonstrates memory efficiency and favorable computation-communication trade-offs, indicating practical suitability for scalable PFCL systems.

Abstract

We focus on the problem of Personalized Federated Continual Learning (PFCL): a group of distributed clients, each with a sequence of local tasks on arbitrary data distributions, collaborate through a central server to train a personalized model at each client, with the model expected to achieve good performance on all local tasks. We propose a novel PFCL framework called Federated Memory Strengthening FedMeS to address the challenges of client drift and catastrophic forgetting. In FedMeS, each client stores samples from previous tasks using a small amount of local memory, and leverages this information to both 1) calibrate gradient updates in training process; and 2) perform KNN-based Gaussian inference to facilitate personalization. FedMeS is designed to be task-oblivious, such that the same inference process is applied to samples from all tasks to achieve good performance. FedMeS is analyzed theoretically and evaluated experimentally. It is shown to outperform all baselines in average accuracy and forgetting rate, over various combinations of datasets, task distributions, and client numbers.

Figures (9)

• Figure 1: (a) Illustration of the model parameter divergence with non-IID datasets. (b) Illustration of catastrophic forgetting. (c) An overview of a PFCL system in IIoT scenario.
• Figure 2: Workflow of FedMeS. Local memory is utilized in both training and inference processes.
• Figure 3: (a - d) Average accuracy and forgetting rate among all clients over 10 tasks on Split CIFAR-100 with 10 and 20 clients; (e - h) Average accuracy and forgetting rate among all clients over 10 tasks on Split MiniImageNet with 10 and 20 clients.
• Figure 4: (a) and (b) Client accuracies averaged over all learned tasks on Split CIFAR-100 with 10 and 20 clients; (c) and (d) Client accuracies averaged over all learned tasks on Split MiniImageNet with 10 and 20 clients. The shaded area is the accuracy range of tasks for each client.
• Figure 5: (a - d) Average accuracy and Average forgetting rate among all clients in all learned tasks at x-th task on Split EMINIST with 10 clients and 20 clients. (e - h) Average accuracy and Average forgetting rate among all clients in all learned tasks at x-th task on Split CORe50 with 10 clients and 20 clients.