One-Shot Federated Learning

TL;DR

This work tackles the high communication and privacy costs of federated learning by replacing iterative rounds with a single communication round in which a central server aggregates complete local models through ensemble methods. It introduces a practical one-shot framework that selects a compact subset of local models via strategies such as cross-validation performance, data-volume thresholds, or random sampling, and optionally uses distillation with unlabeled proxy data to compress the ensemble for efficiency and privacy. Empirically, the approach yields substantial improvements over local baselines (a relative ROC-AUC gain of $51.5\%$) and approaches the unattainable global ideal (within $90.1\%$), across real-world datasets (EMNIST, Sentiment140, Gleam). The paper also outlines future directions including cohort-based personalization, privacy guarantees for distillation, few-shot extensions, and adaptation to non-convex models, highlighting practical pathways for scalable, privacy-aware federated learning.

Abstract

We present one-shot federated learning, where a central server learns a global model over a network of federated devices in a single round of communication. Our approach - drawing on ensemble learning and knowledge aggregation - achieves an average relative gain of 51.5% in AUC over local baselines and comes within 90.1% of the (unattainable) global ideal. We discuss these methods and identify several promising directions of future work.

Figures (4)

• Figure 1: Mean AUC across devices. Ensemble methods consistently outperform the local baseline.
• Figure 1: Summary of federated datasets for empirical study.
• Figure 2: Sent140 device score distribution. Ensemble methods significantly improve over local models.
• Figure 3: Comparison of distilled model and ensemble for different proxy data sizes (averaged over 5 trials). The distilled model matches the original ensemble performance with relatively few proxy samples.