Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Multi-Prototype-Guided Federated Knowledge Distillation Approach in AI-RAN Enabled Multi-Access Edge Computing System

Luyao Zou, Hayoung Oh, Chu Myaet Thwal, Apurba Adhikary, Seohyeon Hong, Zhu Han

TL;DR

A multi-prototype-guided federated knowledge distillation (MP-FedKD) approach is proposed, which outperforms the considered state-of-the-art baselines regarding accuracy, average accuracy and errors (RMSE and MAE).

Abstract

With the development of wireless network, Multi-Access Edge Computing (MEC) and Artificial Intelligence (AI)-native Radio Access Network (RAN) have attracted significant attention. Particularly, the integration of AI-RAN and MEC is envisioned to transform network efficiency and responsiveness. Therefore, it is valuable to investigate AI-RAN enabled MEC system. Federated learning (FL) nowadays is emerging as a promising approach for AI-RAN enabled MEC system, in which edge devices are enabled to train a global model cooperatively without revealing their raw data. However, conventional FL encounters the challenge in processing the non-independent and identically distributed (non-IID) data. Single prototype obtained by averaging the embedding vectors per class can be employed in FL to handle the data heterogeneity issue. Nevertheless, this may result in the loss of useful information owing to the average operation. Therefore, in this paper, a multi-prototype-guided federated knowledge distillation (MP-FedKD) approach is proposed. Particularly, self-knowledge distillation is integrated into FL to deal with the non-IID issue. To cope with the problem of information loss caused by single prototype-based strategy, multi-prototype strategy is adopted, where we present a conditional hierarchical agglomerative clustering (CHAC) approach and a prototype alignment scheme. Additionally, we design a novel loss function (called LEMGP loss) for each local client, where the relationship between global prototypes and local embedding will be focused. Extensive experiments over multiple datasets with various non-IID settings showcase that the proposed MP-FedKD approach outperforms the considered state-of-the-art baselines regarding accuracy, average accuracy and errors (RMSE and MAE).

A Multi-Prototype-Guided Federated Knowledge Distillation Approach in AI-RAN Enabled Multi-Access Edge Computing System

TL;DR

A multi-prototype-guided federated knowledge distillation (MP-FedKD) approach is proposed, which outperforms the considered state-of-the-art baselines regarding accuracy, average accuracy and errors (RMSE and MAE).

Abstract

With the development of wireless network, Multi-Access Edge Computing (MEC) and Artificial Intelligence (AI)-native Radio Access Network (RAN) have attracted significant attention. Particularly, the integration of AI-RAN and MEC is envisioned to transform network efficiency and responsiveness. Therefore, it is valuable to investigate AI-RAN enabled MEC system. Federated learning (FL) nowadays is emerging as a promising approach for AI-RAN enabled MEC system, in which edge devices are enabled to train a global model cooperatively without revealing their raw data. However, conventional FL encounters the challenge in processing the non-independent and identically distributed (non-IID) data. Single prototype obtained by averaging the embedding vectors per class can be employed in FL to handle the data heterogeneity issue. Nevertheless, this may result in the loss of useful information owing to the average operation. Therefore, in this paper, a multi-prototype-guided federated knowledge distillation (MP-FedKD) approach is proposed. Particularly, self-knowledge distillation is integrated into FL to deal with the non-IID issue. To cope with the problem of information loss caused by single prototype-based strategy, multi-prototype strategy is adopted, where we present a conditional hierarchical agglomerative clustering (CHAC) approach and a prototype alignment scheme. Additionally, we design a novel loss function (called LEMGP loss) for each local client, where the relationship between global prototypes and local embedding will be focused. Extensive experiments over multiple datasets with various non-IID settings showcase that the proposed MP-FedKD approach outperforms the considered state-of-the-art baselines regarding accuracy, average accuracy and errors (RMSE and MAE).
Paper Structure (24 sections, 14 equations, 8 figures, 8 tables, 2 algorithms)

This paper contains 24 sections, 14 equations, 8 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. KD-based FL
  4. Single Prototype-based Strategy
  5. Multi-Prototype-based Strategy
  6. System Model
  7. Model of AI-RAN Enabled MEC System
  8. FL Model in AI-RAN Enabled MEC System
  9. Prototype-based Model in AI-RAN Enabled MEC System
  10. Methodology
  11. Self-Knowledge Distillation
  12. CHAC-based Multi-Prototype Generation
  13. A Novel Prototype Alignment (PA)
  14. LEMGP Loss Design
  15. Local Loss Function Design
  16. ...and 9 more sections

Figures (8)

  • Figure 1: System model of the AI-RAN enabled MEC system, where each edge device can generate local prototypes and local model through training.
  • Figure 2: CHAC for per-class multi-prototype generation on the client (left) and architecture of the proposed MP-FedKD (right).
  • Figure 3: Accuracy comparison between K-Means and CHAC (Ours).
  • Figure 4: RMSE and MAE achieved by FedProx, FedProto and the proposed method using various datasets ($Dir=0.9$).
  • Figure 5: (a)-(c) Accuracy comparison among FedAS, MOON, FedProto and the proposed method via various datasets ($Dir=0.9$). (d) illustrates the accuracy achieved by the proposed method with considering different cluster number settings towards M+F dataset.
  • ...and 3 more figures