FPPL: An Efficient and Non-IID Robust Federated Continual Learning Framework

TL;DR

FPPL tackles the challenge of non-IID federated continual learning by coupling lightweight prompt tuning with prototype-driven mechanisms. It operates rehearsal-free: clients learn task-specific prompts while a fusion function leverages accumulated knowledge, and servers perform debiasing and contrastive learning using local-to-global prototypes. The approach achieves strong accuracy and low forgetting across ImageNet-R, CUB-200, and CIFAR-100 with dramatically reduced communication and storage costs compared to rehearsal-based baselines. Overall, FPPL offers a scalable, privacy-preserving FCL framework robust to data heterogeneity in realistic distributed settings.

Abstract

Federated continual learning (FCL) aims to learn from sequential data stream in the decentralized federated learning setting, while simultaneously mitigating the catastrophic forgetting issue in classical continual learning. Existing FCL methods usually employ typical rehearsal mechanisms, which could result in privacy violations or additional onerous storage and computational burdens. In this work, an efficient and non-IID robust federated continual learning framework, called Federated Prototype-Augmented Prompt Learning (FPPL), is proposed. The FPPL can collaboratively learn lightweight prompts augmented by prototypes without rehearsal. On the client side, a fusion function is employed to fully leverage the knowledge contained in task-specific prompts for alleviating catastrophic forgetting. Additionally, global prototypes aggregated from the server are used to obtain unified representation through contrastive learning, mitigating the impact of non-IID-derived data heterogeneity. On the server side, locally uploaded prototypes are utilized to perform debiasing on the classifier, further alleviating the performance degradation caused by both non-IID and catastrophic forgetting. Empirical evaluations demonstrate the effectiveness of FPPL, achieving notable performance with an efficient design while remaining robust to diverse non-IID degrees. Code is available at: https://github.com/ycheoo/FPPL.

Figures (4)

• Figure 1: Illustration of the prompt tuning utilized in our FPPL. The lightweight prompt and classifier parameters are tuned during learning, while achieving parameter-efficient tuning for pre-trained Vision Transformer.
• Figure 2: The overview of proposed FPPL framework. 1) Client: Extract unified representations across different clients by $\mathcal{L}_{ur}$ and construct local prototype set using Eq. \ref{['eq:prototype_local']} after training local model. 2) Server: Debias the classifier of the aggregated parameters by $\mathcal{L}_{debias}$ and construct global prototypes returned to all clients using Eq. \ref{['eq:prototype_global']}. 3) Lightweight prompts, linear parameters and prototypes are transmitted between clients and the server, while no additional data is stored on each client.
• Figure 3: Figure \ref{['fig:incacc']} shows accuracy curves on 20-task ImageNet-R (left) and 20-task CUB-200 (right) during the whole FCL procedure. Figure \ref{['fig:cifar100_avgacc']} presents average accuracy on 10-task CIFAR-100 under IID and non-IID setting.
• Figure 4: Average accuracy on 20-task ImageNet-R (left) and 20-task CUB-200 (right) under different non-IID degrees.