Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gains: Fine-grained Federated Domain Adaptation in Open Set

Zhengyi Zhong, Wenzheng Jiang, Weidong Bao, Ji Wang, Cheems Wang, Guanbo Wang, Yongheng Deng, Ju Ren

TL;DR

Gains addresses open-set federated learning with continuously arriving clients by introducing a fine-grained FDA framework that splits models into an encoder and a classifier to separately detect domain- and class-increment knowledge. It employs fine-grained knowledge discrimination using $Diff^F$ and $Diff^C$ with thresholds to identify new knowledge type, followed by contribution-driven aggregation to rapidly integrate new knowledge while preserving source-domain performance via an anti-forgetting mechanism. The method demonstrates superior target, source, and global accuracy and faster convergence across multiple multi-domain shifts on DigitFive and Amazon Review, including sequential FDA scenarios. These results advance practical FDA in open environments and provide a scalable approach for continual knowledge integration in FL.

Abstract

Conventional federated learning (FL) assumes a closed world with a fixed total number of clients. In contrast, new clients continuously join the FL process in real-world scenarios, introducing new knowledge. This raises two critical demands: detecting new knowledge, i.e., knowledge discovery, and integrating it into the global model, i.e., knowledge adaptation. Existing research focuses on coarse-grained knowledge discovery, and often sacrifices source domain performance and adaptation efficiency. To this end, we propose a fine-grained federated domain adaptation approach in open set (Gains). Gains splits the model into an encoder and a classifier, empirically revealing features extracted by the encoder are sensitive to domain shifts while classifier parameters are sensitive to class increments. Based on this, we develop fine-grained knowledge discovery and contribution-driven aggregation techniques to identify and incorporate new knowledge. Additionally, an anti-forgetting mechanism is designed to preserve source domain performance, ensuring balanced adaptation. Experimental results on multi-domain datasets across three typical data-shift scenarios demonstrate that Gains significantly outperforms other baselines in performance for both source-domain and target-domain clients. Code is available at: https://github.com/Zhong-Zhengyi/Gains.

Gains: Fine-grained Federated Domain Adaptation in Open Set

TL;DR

Gains addresses open-set federated learning with continuously arriving clients by introducing a fine-grained FDA framework that splits models into an encoder and a classifier to separately detect domain- and class-increment knowledge. It employs fine-grained knowledge discrimination using and with thresholds to identify new knowledge type, followed by contribution-driven aggregation to rapidly integrate new knowledge while preserving source-domain performance via an anti-forgetting mechanism. The method demonstrates superior target, source, and global accuracy and faster convergence across multiple multi-domain shifts on DigitFive and Amazon Review, including sequential FDA scenarios. These results advance practical FDA in open environments and provide a scalable approach for continual knowledge integration in FL.

Abstract

Conventional federated learning (FL) assumes a closed world with a fixed total number of clients. In contrast, new clients continuously join the FL process in real-world scenarios, introducing new knowledge. This raises two critical demands: detecting new knowledge, i.e., knowledge discovery, and integrating it into the global model, i.e., knowledge adaptation. Existing research focuses on coarse-grained knowledge discovery, and often sacrifices source domain performance and adaptation efficiency. To this end, we propose a fine-grained federated domain adaptation approach in open set (Gains). Gains splits the model into an encoder and a classifier, empirically revealing features extracted by the encoder are sensitive to domain shifts while classifier parameters are sensitive to class increments. Based on this, we develop fine-grained knowledge discovery and contribution-driven aggregation techniques to identify and incorporate new knowledge. Additionally, an anti-forgetting mechanism is designed to preserve source domain performance, ensuring balanced adaptation. Experimental results on multi-domain datasets across three typical data-shift scenarios demonstrate that Gains significantly outperforms other baselines in performance for both source-domain and target-domain clients. Code is available at: https://github.com/Zhong-Zhengyi/Gains.
Paper Structure (19 sections, 15 equations, 5 figures, 8 tables, 1 algorithm)

This paper contains 19 sections, 15 equations, 5 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Methodology
  4. Fine-grained knowledge discrimination
  5. Contribution-driven knowledge adaptation
  6. Algorithm
  7. Experimental verification
  8. Experiment setting
  9. New knowledge discovery
  10. Knowledge adaptation
  11. Computing complexity analysis.
  12. Sensitivity analysis of the thresholds
  13. Conclusion and discussion
  14. Motivation experiment settings
  15. Experimental details
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Challenge discription.
  • Figure 2: Differences in the encoder (left), classifier (middle), and extracted feature values (right) when the new client carries different types of knowledge.
  • Figure 3: Gains consists of two phases: knowledge discovery (upper) and knowledge adaptation (lower). The former works on identifying the type of new knowledge, while the latter attempts to achieve rapid integration of new knowledge and strike a balance between new and old knowledge.
  • Figure 4: Training process of DigitFive under different data shift scenarios.
  • Figure 5: Training process of Amazon Review under different data shift scenarios.