Federated Continual Learning for Edge-AI: A Comprehensive Survey

TL;DR

This survey thoroughly review the state-of-the-art research and presents the first comprehensive survey of FCL for Edge-AI, categorizing FCL methods based on three task characteristics: federated class continual learning, federated domain continual learning, and federated task continual learning.

Abstract

Edge-AI, the convergence of edge computing and artificial intelligence (AI), has become a promising paradigm that enables the deployment of advanced AI models at the network edge, close to users. In Edge-AI, federated continual learning (FCL) has emerged as an imperative framework, which fuses knowledge from different clients while preserving data privacy and retaining knowledge from previous tasks as it learns new ones. By so doing, FCL aims to ensure stable and reliable performance of learning models in dynamic and distributed environments. In this survey, we thoroughly review the state-of-the-art research and present the first comprehensive survey of FCL for Edge-AI. We categorize FCL methods based on three task characteristics: federated class continual learning, federated domain continual learning, and federated task continual learning. For each category, an in-depth investigation and review of the representative methods are provided, covering background, challenges, problem formalisation, solutions, and limitations. Besides, existing real-world applications empowered by FCL are reviewed, indicating the current progress and potential of FCL in diverse application domains. Furthermore, we discuss and highlight several prospective research directions of FCL such as algorithm-hardware co-design for FCL and FCL with foundation models, which could provide insights into the future development and practical deployment of FCL in the era of Edge-AI.

Figures (4)

• Figure 1: An overview of our federated continual learning survey
• Figure 2: Diagram of the relation among FCCL methods. There are five categories in our paper: generative replay (purple), parameter regularization (yellow), parameter decomposition (orange), prompt-based methods (green) and knowledge distillation (blue). Auxiliary datasets, the data-free manner with distillation and rehearsal-free methods, frequently employed in some methods, are indicated by the dashed boxes.
• Figure 3: Overall diagram of challenges faced by FDCL, challenge 1: privacy protection of multi-source domains and domain drift of the global model concerning the local model (inter-domain); challenge 2: generalization of the global model for the unknown domain, and domain drift of locally known domains over time and data (intra-domain).
• Figure 4: The elaborated taxonomy of representative federated task continual learning methods