Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PIP: Prototypes-Injected Prompt for Federated Class Incremental Learning

Muhammad Anwar Ma'sum, Mahardhika Pratama, Savitha Ramasamy, Lin Liu, Habibullah Habibullah, Ryszard Kowalczyk

TL;DR

The paper tackles FCIL by removing the need to share large backbones or raw data, proposing a memory-efficient approach that combines prototype-injected prompts with cross-client prototypes and server-side weighted Gaussian aggregation. The method, PIP, comprises local prompt learning to combat forgetting, a shared Gaussian prototype injection to harmonize non-IID client data, prototype augmentation to address class imbalance, and a principled Gaussian-weighted aggregation to improve generalization. Theoretical convergence and generalization guarantees accompany extensive experiments on CIFAR100, MiniImageNet, and TinyImageNet, where PIP-DualP achieves state-of-the-art accuracy and lower forgetting compared to baselines, with strong robustness to task size, client participation, and rounds. These results suggest that memory-efficient prompt-based FCIL with prototype-based cross-client knowledge transfer offers practical applicability for privacy-preserving, scalable continual learning in distributed settings.

Abstract

Federated Class Incremental Learning (FCIL) is a new direction in continual learning (CL) for addressing catastrophic forgetting and non-IID data distribution simultaneously. Existing FCIL methods call for high communication costs and exemplars from previous classes. We propose a novel rehearsal-free method for FCIL named prototypes-injected prompt (PIP) that involves 3 main ideas: a) prototype injection on prompt learning, b) prototype augmentation, and c) weighted Gaussian aggregation on the server side. Our experiment result shows that the proposed method outperforms the current state of the arts (SOTAs) with a significant improvement (up to 33%) in CIFAR100, MiniImageNet and TinyImageNet datasets. Our extensive analysis demonstrates the robustness of PIP in different task sizes, and the advantage of requiring smaller participating local clients, and smaller global rounds. For further study, source codes of PIP, baseline, and experimental logs are shared publicly in https://github.com/anwarmaxsum/PIP.

PIP: Prototypes-Injected Prompt for Federated Class Incremental Learning

TL;DR

The paper tackles FCIL by removing the need to share large backbones or raw data, proposing a memory-efficient approach that combines prototype-injected prompts with cross-client prototypes and server-side weighted Gaussian aggregation. The method, PIP, comprises local prompt learning to combat forgetting, a shared Gaussian prototype injection to harmonize non-IID client data, prototype augmentation to address class imbalance, and a principled Gaussian-weighted aggregation to improve generalization. Theoretical convergence and generalization guarantees accompany extensive experiments on CIFAR100, MiniImageNet, and TinyImageNet, where PIP-DualP achieves state-of-the-art accuracy and lower forgetting compared to baselines, with strong robustness to task size, client participation, and rounds. These results suggest that memory-efficient prompt-based FCIL with prototype-based cross-client knowledge transfer offers practical applicability for privacy-preserving, scalable continual learning in distributed settings.

Abstract

Federated Class Incremental Learning (FCIL) is a new direction in continual learning (CL) for addressing catastrophic forgetting and non-IID data distribution simultaneously. Existing FCIL methods call for high communication costs and exemplars from previous classes. We propose a novel rehearsal-free method for FCIL named prototypes-injected prompt (PIP) that involves 3 main ideas: a) prototype injection on prompt learning, b) prototype augmentation, and c) weighted Gaussian aggregation on the server side. Our experiment result shows that the proposed method outperforms the current state of the arts (SOTAs) with a significant improvement (up to 33%) in CIFAR100, MiniImageNet and TinyImageNet datasets. Our extensive analysis demonstrates the robustness of PIP in different task sizes, and the advantage of requiring smaller participating local clients, and smaller global rounds. For further study, source codes of PIP, baseline, and experimental logs are shared publicly in https://github.com/anwarmaxsum/PIP.
Paper Structure (31 sections, 89 equations, 3 figures, 17 tables, 1 algorithm)

This paper contains 31 sections, 89 equations, 3 figures, 17 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Problem Formulation
  5. Proposed Approach and Baselines: Federated Prompt-Based Continual Learning for FCIL
  6. Proposed Method: Prototype-Injected Prompt (PIP)
  7. Handling Local Catastrophic Forgetting via Prompt Learning
  8. Shared Prototype Injection: What, Why and How?
  9. Handling Class Imbalance via Prototype Augmentation
  10. Server Weighted Aggregation
  11. Theoretical Analysis
  12. Experiment Results and Analysis
  13. Experimental Setting
  14. Numerical Results
  15. Forgetting Analysis
  16. ...and 16 more sections

Figures (3)

  • Figure 1: Visualization of our proposed method (PIP) that includes prototypes-injected-prompt learning with $\mathcal{L}_{l+}$ to handle local catastrophic forgetting, shared prototypes to handle non-i.i.d distributions between clients, prototypes augmentation to handle class-imbalance, and server Gaussian weighted aggregation to improve global model generalizations.
  • Figure 2: Performance of the consolidated methods in CIFAR100, MiniImageNet and TinyImageNet with T=5 and T=20.
  • Figure 3: Performance of the consolidated methods in CIFAR100 and TinyImageNet with smaller local clients (L) and smaller rounds.