Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Personalized Federated Domain-Incremental Learning based on Adaptive Knowledge Matching

Yichen Li, Wenchao Xu, Haozhao Wang, Ruixuan Li, Yining Qi, Jingcai Guo

TL;DR

This work addresses catastrophic forgetting in Federated Domain-Incremental Learning by introducing pFedDIL, a personalized, adaptive framework that uses auxiliary classifiers to measure knowledge correlations between tasks. When a new task arrives, clients selectively reuse similar personalized models or initialize anew, guided by a knowledge matching intensity and augmented by a knowledge migration loss that progressively transfers relevant knowledge. A key innovation is sharing partial parameters between the auxiliary classifiers and the main models to condense parameters while enabling effective correlation-based ensemble inference. Experiments on Digit-10, Office-31, and DomainNet show that pFedDIL consistently improves average task accuracy by up to 14.35% over strong baselines, with ablations confirming the importance of migration, sharing, and correlation signaling for robust FDIL performance.

Abstract

This paper focuses on Federated Domain-Incremental Learning (FDIL) where each client continues to learn incremental tasks where their domain shifts from each other. We propose a novel adaptive knowledge matching-based personalized FDIL approach (pFedDIL) which allows each client to alternatively utilize appropriate incremental task learning strategy on the correlation with the knowledge from previous tasks. More specifically, when a new task arrives, each client first calculates its local correlations with previous tasks. Then, the client can choose to adopt a new initial model or a previous model with similar knowledge to train the new task and simultaneously migrate knowledge from previous tasks based on these correlations. Furthermore, to identify the correlations between the new task and previous tasks for each client, we separately employ an auxiliary classifier to each target classification model and propose sharing partial parameters between the target classification model and the auxiliary classifier to condense model parameters. We conduct extensive experiments on several datasets of which results demonstrate that pFedDIL outperforms state-of-the-art methods by up to 14.35\% in terms of average accuracy of all tasks.

Personalized Federated Domain-Incremental Learning based on Adaptive Knowledge Matching

TL;DR

This work addresses catastrophic forgetting in Federated Domain-Incremental Learning by introducing pFedDIL, a personalized, adaptive framework that uses auxiliary classifiers to measure knowledge correlations between tasks. When a new task arrives, clients selectively reuse similar personalized models or initialize anew, guided by a knowledge matching intensity and augmented by a knowledge migration loss that progressively transfers relevant knowledge. A key innovation is sharing partial parameters between the auxiliary classifiers and the main models to condense parameters while enabling effective correlation-based ensemble inference. Experiments on Digit-10, Office-31, and DomainNet show that pFedDIL consistently improves average task accuracy by up to 14.35% over strong baselines, with ablations confirming the importance of migration, sharing, and correlation signaling for robust FDIL performance.

Abstract

This paper focuses on Federated Domain-Incremental Learning (FDIL) where each client continues to learn incremental tasks where their domain shifts from each other. We propose a novel adaptive knowledge matching-based personalized FDIL approach (pFedDIL) which allows each client to alternatively utilize appropriate incremental task learning strategy on the correlation with the knowledge from previous tasks. More specifically, when a new task arrives, each client first calculates its local correlations with previous tasks. Then, the client can choose to adopt a new initial model or a previous model with similar knowledge to train the new task and simultaneously migrate knowledge from previous tasks based on these correlations. Furthermore, to identify the correlations between the new task and previous tasks for each client, we separately employ an auxiliary classifier to each target classification model and propose sharing partial parameters between the target classification model and the auxiliary classifier to condense model parameters. We conduct extensive experiments on several datasets of which results demonstrate that pFedDIL outperforms state-of-the-art methods by up to 14.35\% in terms of average accuracy of all tasks.
Paper Structure (10 sections, 8 equations, 3 figures, 5 tables, 1 algorithm)

This paper contains 10 sections, 8 equations, 3 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Definition
  5. The Proposed pFedDIL Approach
  6. Experiments
  7. Setup
  8. Performance Overview
  9. Limitations
  10. Conclusion and Future Work

Figures (3)

  • Figure 1: Illustration of the pFedDIL framework. In the training stage, each client randomly receives one domain dataset without replacement to form a task, where different colors represent different domains. Clients can adaptively employ an appropriate incremental task learning strategy and migrate the knowledge from previous tasks with the knowledge matching intensity.
  • Figure 2: Visualized performance w.r.t data heterogeneity for three datasets.
  • Figure 3: Performance of pFedDIL under different configurations. Here, we select three general parameters in the FL setting: (a) local training epoch $E$, (b) sample size $B$ in the classifier, (c) client selection ratio $r$ of all clients on DomainNet with $\alpha$ = 10.