Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Meta-Learning with Fast Convergence and Efficient Communication

Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, Xiuqiang He

TL;DR

The paper introduces FedMeta, a federated meta-learning framework that replaces a shared global model with a shared parameterized algorithm (e.g., MAML or Meta-SGD) to address privacy and non-IID data in distributed edge environments. Clients receive algorithm parameters, train locally on support data, and report losses to update the meta-learner, enabling rapid adaptation with reduced communication. Across LEAF benchmarks and a production dataset, FedMeta achieves 2.82-4.33x lower communication and 3.23-14.84% higher accuracy versus FedAvg, with faster convergence. These results demonstrate the practical value of meta-learning in federated settings for personalized, privacy-preserving, and communication-efficient edge AI tasks.

Abstract

Statistical and systematic challenges in collaboratively training machine learning models across distributed networks of mobile devices have been the bottlenecks in the real-world application of federated learning. In this work, we show that meta-learning is a natural choice to handle these issues, and propose a federated meta-learning framework FedMeta, where a parameterized algorithm (or meta-learner) is shared, instead of a global model in previous approaches. We conduct an extensive empirical evaluation on LEAF datasets and a real-world production dataset, and demonstrate that FedMeta achieves a reduction in required communication cost by 2.82-4.33 times with faster convergence, and an increase in accuracy by 3.23%-14.84% as compared to Federated Averaging (FedAvg) which is a leading optimization algorithm in federated learning. Moreover, FedMeta preserves user privacy since only the parameterized algorithm is transmitted between mobile devices and central servers, and no raw data is collected onto the servers.

Federated Meta-Learning with Fast Convergence and Efficient Communication

TL;DR

The paper introduces FedMeta, a federated meta-learning framework that replaces a shared global model with a shared parameterized algorithm (e.g., MAML or Meta-SGD) to address privacy and non-IID data in distributed edge environments. Clients receive algorithm parameters, train locally on support data, and report losses to update the meta-learner, enabling rapid adaptation with reduced communication. Across LEAF benchmarks and a production dataset, FedMeta achieves 2.82-4.33x lower communication and 3.23-14.84% higher accuracy versus FedAvg, with faster convergence. These results demonstrate the practical value of meta-learning in federated settings for personalized, privacy-preserving, and communication-efficient edge AI tasks.

Abstract

Statistical and systematic challenges in collaboratively training machine learning models across distributed networks of mobile devices have been the bottlenecks in the real-world application of federated learning. In this work, we show that meta-learning is a natural choice to handle these issues, and propose a federated meta-learning framework FedMeta, where a parameterized algorithm (or meta-learner) is shared, instead of a global model in previous approaches. We conduct an extensive empirical evaluation on LEAF datasets and a real-world production dataset, and demonstrate that FedMeta achieves a reduction in required communication cost by 2.82-4.33 times with faster convergence, and an increase in accuracy by 3.23%-14.84% as compared to Federated Averaging (FedAvg) which is a leading optimization algorithm in federated learning. Moreover, FedMeta preserves user privacy since only the parameterized algorithm is transmitted between mobile devices and central servers, and no raw data is collected onto the servers.

Paper Structure

This paper contains 15 sections, 3 equations, 7 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Federated Meta-Learning
  4. The Meta-Learning Approach
  5. The Federated Meta-Learning Framework
  6. Experiments
  7. Evaluation Scheme
  8. LEAF Datasets
  9. Real Industrial Recommendation Task
  10. Conclusion and Future Work
  11. Experimental Details
  12. Datasets and Models
  13. Implementation Details
  14. Additional Experiments on LEAF
  15. Additional Experiments on Industrial Task

Figures (7)

  • Figure 1: Workflow of the federated meta-learning framework.
  • Figure 2: Performance on LEAF datasets for FedAvg and three running examples of FedMeta. The support fraction setting for all datasets is 20%. Compared with intuitive FedAvg, all the running examples within FedMeta framework provide faster convergence and higher accuracy.
  • Figure 3: System overhead for achieving a target accuracy in different methods.The target accuracies for FEMNIST, Shakespeare and Sent140 are 74%, 38% and 70% respectively.
  • Figure 4: Performance on LEAF datasets for FedAvg and three running examples of FedMeta. The support fraction setting for all datasets is 50%
  • Figure 5: Performance on LEAF datasets for FedAvg and three running examples of FedMeta. The support fraction setting for all datasets is 90%
  • ...and 2 more figures