Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning

TL;DR

This paper addresses how federated learning can be enhanced by leveraging model fusion techniques and by integrating with other learning paradigms (Federated X Learning). It provides a taxonomy of four main fusion approaches (adaptive weighting, attentive aggregation, regularization, clustering) plus Bayesian methods, and surveys a wide range of Federated X Learning paradigms including transfer, knowledge distillation, multi-task, meta-learning, adversarial, semi-supervised, unsupervised, and reinforcement learning. The analysis highlights persistent challenges such as statistical and model heterogeneity, security, and communication efficiency, and connects these methods to real-world applications in recommendation systems and healthcare, while outlining practical future directions. The work offers a structured lens to reason about combining FL with diverse learning objectives and proposes future benchmarking and production-oriented research to accelerate adoption.

Abstract

Federated learning is a new learning paradigm that decouples data collection and model training via multi-party computation and model aggregation. As a flexible learning setting, federated learning has the potential to integrate with other learning frameworks. We conduct a focused survey of federated learning in conjunction with other learning algorithms. Specifically, we explore various learning algorithms to improve the vanilla federated averaging algorithm and review model fusion methods such as adaptive aggregation, regularization, clustered methods, and Bayesian methods. Following the emerging trends, we also discuss federated learning in the intersection with other learning paradigms, termed federated X learning, where X includes multitask learning, meta-learning, transfer learning, unsupervised learning, and reinforcement learning. In addition to reviewing state-of-the-art studies, this paper also identifies key challenges and applications in this field, while also highlighting promising future directions.