SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devices

TL;DR

The paper tackles data drift and privacy constraints in on-device continual learning by introducing SacFL, a self-adaptive federated continual learning framework that splits models into a task-robust Encoder and a lightweight task-sensitive Decoder to dramatically reduce on-device storage. It adds a memory-efficient, label-free data-drift detector based on contrastive learning and a defense mechanism for adversarial tasks, enabling devices to autonomously trigger continual learning or defensive measures. The approach is validated across image and text datasets for class- and domain-incremental scenarios, showing superior performance and substantial resource savings, including a real-world demo system. Overall, SacFL offers practical, privacy-preserving, self-adaptive continual learning for resource-constrained end devices with robustness to adversarial inputs and strong empirical gains over existing methods.

Abstract

The proliferation of end devices has led to a distributed computing paradigm, wherein on-device machine learning models continuously process diverse data generated by these devices. The dynamic nature of this data, characterized by continuous changes or data drift, poses significant challenges for on-device models. To address this issue, continual learning (CL) is proposed, enabling machine learning models to incrementally update their knowledge and mitigate catastrophic forgetting. However, the traditional centralized approach to CL is unsuitable for end devices due to privacy and data volume concerns. In this context, federated continual learning (FCL) emerges as a promising solution, preserving user data locally while enhancing models through collaborative updates. Aiming at the challenges of limited storage resources for CL, poor autonomy in task shift detection, and difficulty in coping with new adversarial tasks in FCL scenario, we propose a novel FCL framework named SacFL. SacFL employs an Encoder-Decoder architecture to separate task-robust and task-sensitive components, significantly reducing storage demands by retaining lightweight task-sensitive components for resource-constrained end devices. Moreover, $\rm{SacFL}$ leverages contrastive learning to introduce an autonomous data shift detection mechanism, enabling it to discern whether a new task has emerged and whether it is a benign task. This capability ultimately allows the device to autonomously trigger CL or attack defense strategy without additional information, which is more practical for end devices. Comprehensive experiments conducted on multiple text and image datasets, such as Cifar100 and THUCNews, have validated the effectiveness of $\rm{SacFL}$ in both class-incremental and domain-incremental scenarios. Furthermore, a demo system has been developed to verify its practicality.

Figures (12)

• Figure 1: The changes of parameters in different model layers during the training process. It is worth noting that each task is trained for 50 iterations, and there is no need to calculate the changes in model parameters for the 0th task. Therefore, the abscissa in the figure starts from 50. The vertical axis represents the difference between specific layer parameters and the corresponding layer parameters after training the 0th task.
• Figure 2: The framework of $\rm{SacFL}$. When no task change occurs, the client trains the Encoder and Decoder using the classical FL approach, with the exception of performing data drift detection during each iteration. If data drift is detected, the Decoder from the previous task is pushed to the local Decoder pool. At the same time, the updated Encoder and Decoder are uploaded to the server to determine whether the new task is adversarial. If an adversarial task is detected, local attack defense mechanisms and Krum aggregation are activated to mitigate the impact of the attack, continuing until the next task is identified.
• Figure 3: Class-incremental data setting. Taking 5 clients as an example, the data of each class is divided into 5 parts. Through random selection, Client 1 extracts the 1st, 4th, and 5th parts from the data labeled 0, 3, and 8 as the data for Task 0. Subsequent tasks are generated in the same manner.
• Figure 4: The change of the feature values extracted by the Encoder.
• Figure 5: Average degradation rate of historical tasks under adversarial attacks.

Theorems & Definitions (1)

• Definition 1