FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence

TL;DR

FedCache addresses the need for efficient personalization in Federated Learning for Edge Intelligence by introducing a server-side knowledge cache that stores sample-associated knowledge. Clients retrieve R-related knowledge via a hash-based encoding and perform ensemble distillation on their local models, enabling sample-grained logit exchange without any public data and with asynchronous updates. The method achieves competitive MAUA compared to state-of-the-art PFL approaches while reducing communication overhead by approximately two orders of magnitude, and supports heterogeneous on-device models. This architecture has practical implications for scalable, privacy-preserving personalized AI at the edge, particularly in systems with limited bandwidth and diverse device capabilities.

Abstract

Edge Intelligence (EI) allows Artificial Intelligence (AI) applications to run at the edge, where data analysis and decision-making can be performed in real-time and close to data sources. To protect data privacy and unify data silos among end devices in EI, Federated Learning (FL) is proposed for collaborative training of shared AI models across devices without compromising data privacy. However, the prevailing FL approaches cannot guarantee model generalization and adaptation on heterogeneous clients. Recently, Personalized Federated Learning (PFL) has drawn growing awareness in EI, as it enables a productive balance between local-specific training requirements inherent in devices and global-generalized optimization objectives for satisfactory performance. However, most existing PFL methods are based on the Parameters Interaction-based Architecture (PIA) represented by FedAvg, which causes unaffordable communication burdens due to large-scale parameters transmission between devices and the edge server. In contrast, Logits Interaction-based Architecture (LIA) allows to update model parameters with logits transfer and gains the advantages of communication lightweight and heterogeneous on-device model allowance compared to PIA. Nevertheless, previous LIA methods attempt to achieve satisfactory performance either relying on unrealistic public datasets or increasing communication overhead for additional information transmission other than logits. To tackle this dilemma, we propose a knowledge cache-driven PFL architecture, named FedCache, which reserves a knowledge cache on the server for fetching personalized knowledge from the samples with similar hashes to each given on-device sample. During the training phase, ensemble distillation is applied to on-device models for constructive optimization with personalized knowledge transferred from the server-side knowledge cache.

Figures (6)

• Figure 1: Schematic diagram of personalized federated learning for edge intelligence.
• Figure 2: Functional module diagram of FedCache.
• Figure 3: Sample matching results on FashionMNIST dataset with $R=3$.
• Figure 4: Overview of executing procedure of FedCache. (1) Hash encoding and uploading. (2) Knowledge cache initialization. (3) Knowledge extraction and uploading. (4) Knowledge fetching. (5) Knowledge ensemble and distributing. (6) Knowledge update. (7) Knowledge acceptance and distillation.
• Figure 5: MAUA (%) per unit of communication overhead in experiments with homogeneous models. Dashed lines indicate the extension of algorithms beyond convergence to the maximum MAUA over communication overheads. The same as below.