FedCache 2.0: Federated Edge Learning with Knowledge Caching and Dataset Distillation

TL;DR

FedCache 2.0 is introduced, a novel personalized FEL architecture that simultaneously addresses challenges related to device constraints and device-server interactions and significantly outperforms state-of-the-art methods regardless of model structures, data distributions, and modalities.

Abstract

Federated Edge Learning (FEL) has emerged as a promising approach for enabling edge devices to collaboratively train machine learning models while preserving data privacy. Despite its advantages, practical FEL deployment faces significant challenges related to device constraints and device-server interactions, necessitating heterogeneous, user-adaptive model training with limited and uncertain communication. In this paper, we introduce FedCache 2.0, a novel personalized FEL architecture that simultaneously addresses these challenges. FedCache 2.0 incorporates the benefits of both dataset distillation and knowledge cache-driven federated learning by storing and organizing distilled data as knowledge in the server-side knowledge cache. Moreover, a device-centric cache sampling strategy is introduced to tailor transferred knowledge for individual devices within controlled communication bandwidth. Extensive experiments on five datasets covering image recognition, audio understanding, and mobile sensor data mining tasks demonstrate that (1) FedCache 2.0 significantly outperforms state-of-the-art methods regardless of model structures, data distributions, and modalities. (2) FedCache 2.0 can train splendid personalized on-device models with at least $\times$28.6 improvement in communication efficiency.

Figures (7)

• Figure 1: Comparison of FedCache 2.0 with state-of-the-art methods in terms of performance and communication efficiency. FedCache 2.0 demonstrates significant performance improvements over state-of-the-art methods with substantially reduced communication overhead compared with model aggregation-based FL methods (MTFL mills2021multi, kNN-Per marfoq2022personalized, and SCDPFLchen2024spectral), as indicated by its position in the top-right of the figure.
• Figure 2: Comparison of FedCache and FedCache2.0. Results in (c) are derived on the CIFAR-10 dataset, taking $\alpha=0.5$ and $K=100$.
• Figure 3: Overview of FedCache 2.0.
• Figure 4: Illustration of varying degrees of data heterogeneity with different $\alpha$ across 10 clients over CIFAR-10 dataset. Each cell's color represents the proportion of samples in their respective datasets.
• Figure 5: Average UA UA (%) per unit of communication cost over image recognition tasks.