Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning with New Knowledge: Fundamentals, Advances, and Futures

Lixu Wang, Yang Zhao, Jiahua Dong, Ating Yin, Qinbin Li, Xiao Wang, Dusit Niyato, Qi Zhu

TL;DR

The paper addresses the challenge of sustaining Federated Learning (FL) as real-world demands introduce new knowledge in the form of unseen features, tasks, models, and algorithms. It formalizes FL as $F(\mathcal{X}, \mathcal{Y}, \mathcal{M}, \mathcal{A})$ with a lifespan $T$ and defines what constitutes new knowledge, then surveys how to incorporate it across four axes: new features (FDG, FODD, FDA), new tasks (task-personalized FL, SSFL), and their combination (federated continual learning), plus considerations for new models and aggregation algorithms and associated threats. The contribution is a first comprehensive taxonomy and analysis of methods to extend FL’s lifespan, including efficiency and security considerations, and it points to an updating repository for ongoing work. The work aims to guide the design of FL systems that can adapt to changing environments while controlling costs and preserving privacy, ultimately enabling practical, long-lived FL deployments across domains.

Abstract

Federated Learning (FL) is a privacy-preserving distributed learning approach that is rapidly developing in an era where privacy protection is increasingly valued. It is this rapid development trend, along with the continuous emergence of new demands for FL in the real world, that prompts us to focus on a very important problem: Federated Learning with New Knowledge. The primary challenge here is to effectively incorporate various new knowledge into existing FL systems and evolve these systems to reduce costs, extend their lifespan, and facilitate sustainable development. In this paper, we systematically define the main sources of new knowledge in FL, including new features, tasks, models, and algorithms. For each source, we thoroughly analyze and discuss how to incorporate new knowledge into existing FL systems and examine the impact of the form and timing of new knowledge arrival on the incorporation process. Furthermore, we comprehensively discuss the potential future directions for FL with new knowledge, considering a variety of factors such as scenario setups, efficiency, and security. There is also a continuously updating repository for this topic: https://github.com/conditionWang/FLNK.

Federated Learning with New Knowledge: Fundamentals, Advances, and Futures

TL;DR

The paper addresses the challenge of sustaining Federated Learning (FL) as real-world demands introduce new knowledge in the form of unseen features, tasks, models, and algorithms. It formalizes FL as with a lifespan and defines what constitutes new knowledge, then surveys how to incorporate it across four axes: new features (FDG, FODD, FDA), new tasks (task-personalized FL, SSFL), and their combination (federated continual learning), plus considerations for new models and aggregation algorithms and associated threats. The contribution is a first comprehensive taxonomy and analysis of methods to extend FL’s lifespan, including efficiency and security considerations, and it points to an updating repository for ongoing work. The work aims to guide the design of FL systems that can adapt to changing environments while controlling costs and preserving privacy, ultimately enabling practical, long-lived FL deployments across domains.

Abstract

Federated Learning (FL) is a privacy-preserving distributed learning approach that is rapidly developing in an era where privacy protection is increasingly valued. It is this rapid development trend, along with the continuous emergence of new demands for FL in the real world, that prompts us to focus on a very important problem: Federated Learning with New Knowledge. The primary challenge here is to effectively incorporate various new knowledge into existing FL systems and evolve these systems to reduce costs, extend their lifespan, and facilitate sustainable development. In this paper, we systematically define the main sources of new knowledge in FL, including new features, tasks, models, and algorithms. For each source, we thoroughly analyze and discuss how to incorporate new knowledge into existing FL systems and examine the impact of the form and timing of new knowledge arrival on the incorporation process. Furthermore, we comprehensively discuss the potential future directions for FL with new knowledge, considering a variety of factors such as scenario setups, efficiency, and security. There is also a continuously updating repository for this topic: https://github.com/conditionWang/FLNK.
Paper Structure (19 sections, 2 equations, 1 figure, 1 table)

This paper contains 19 sections, 2 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Fundamentals and Taxonomy
  3. Incorporate New Features
  4. Federated Domain Generalization
  5. Federated Out-of-Distribution Detection
  6. Federated Domain Adaptation
  7. Incorporate New Tasks
  8. Task-Personalized Federated Learning
  9. Self-Supervised Federated Learning
  10. Incorporate New Tasks with New Features
  11. Federated Continual Learning
  12. Methods
  13. Evaluation
  14. Future Directions
  15. Future Opportunities of Incorporating New Features and Tasks into FL
  16. ...and 4 more sections

Figures (1)

  • Figure 1: Overview of an FL system with new knowledge from different sources. Different types of clients encounter new features and tasks over time, which reflect new demands for FL systems, e.g., client $\mathcal{C}_{k_2}$ needs to deal with the night scenes and conduct segmentation when snowing, and client $\mathcal{C}_{k_3}$ joins FL with the need of handling night scenes and deraining when raining. From a global view, new more advanced models with better architecture (Transformers) and larger sizes (GPT 4) are also needed to incorporate. Besides, new algorithms with better performance (Scaffold) and security guarantees (SecAgg) should be continuously employed in FL as well.