Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedRFQ: Prototype-Based Federated Learning with Reduced Redundancy, Minimal Failure, and Enhanced Quality

Biwei Yan, Hongliang Zhang, Minghui Xu, Dongxiao Yu, Xiuzhen Cheng

TL;DR

This work tackles prototype-based federated learning under non-IID data, where prototype redundancy, prototype failure, and susceptibility to poisoning degrade performance. It introduces FedRFQ, which combines SoftPool pooling to compress and stabilize prototypes with a Byzantine Fault Tolerant detectable aggregation (BFT-detect) to secure the prototype consensus against malicious clients and server malfunctions. The authors provide convergence-oriented analysis and demonstrate, across MNIST, FEMNIST, and CIFAR-10, that FedRFQ achieves higher accuracy with reduced communication and improved robustness compared to baselines. The approach offers practical impact for secure, efficient, and personalized federated learning in heterogeneous, potentially adversarial environments.

Abstract

Federated learning is a powerful technique that enables collaborative learning among different clients. Prototype-based federated learning is a specific approach that improves the performance of local models under non-IID (non-Independently and Identically Distributed) settings by integrating class prototypes. However, prototype-based federated learning faces several challenges, such as prototype redundancy and prototype failure, which limit its accuracy. It is also susceptible to poisoning attacks and server malfunctions, which can degrade the prototype quality. To address these issues, we propose FedRFQ, a prototype-based federated learning approach that aims to reduce redundancy, minimize failures, and improve \underline{q}uality. FedRFQ leverages a SoftPool mechanism, which effectively mitigates prototype redundancy and prototype failure on non-IID data. Furthermore, we introduce the BFT-detect, a BFT (Byzantine Fault Tolerance) detectable aggregation algorithm, to ensure the security of FedRFQ against poisoning attacks and server malfunctions. Finally, we conduct experiments on three different datasets, namely MNIST, FEMNIST, and CIFAR-10, and the results demonstrate that FedRFQ outperforms existing baselines in terms of accuracy when handling non-IID data.

FedRFQ: Prototype-Based Federated Learning with Reduced Redundancy, Minimal Failure, and Enhanced Quality

TL;DR

This work tackles prototype-based federated learning under non-IID data, where prototype redundancy, prototype failure, and susceptibility to poisoning degrade performance. It introduces FedRFQ, which combines SoftPool pooling to compress and stabilize prototypes with a Byzantine Fault Tolerant detectable aggregation (BFT-detect) to secure the prototype consensus against malicious clients and server malfunctions. The authors provide convergence-oriented analysis and demonstrate, across MNIST, FEMNIST, and CIFAR-10, that FedRFQ achieves higher accuracy with reduced communication and improved robustness compared to baselines. The approach offers practical impact for secure, efficient, and personalized federated learning in heterogeneous, potentially adversarial environments.

Abstract

Federated learning is a powerful technique that enables collaborative learning among different clients. Prototype-based federated learning is a specific approach that improves the performance of local models under non-IID (non-Independently and Identically Distributed) settings by integrating class prototypes. However, prototype-based federated learning faces several challenges, such as prototype redundancy and prototype failure, which limit its accuracy. It is also susceptible to poisoning attacks and server malfunctions, which can degrade the prototype quality. To address these issues, we propose FedRFQ, a prototype-based federated learning approach that aims to reduce redundancy, minimize failures, and improve \underline{q}uality. FedRFQ leverages a SoftPool mechanism, which effectively mitigates prototype redundancy and prototype failure on non-IID data. Furthermore, we introduce the BFT-detect, a BFT (Byzantine Fault Tolerance) detectable aggregation algorithm, to ensure the security of FedRFQ against poisoning attacks and server malfunctions. Finally, we conduct experiments on three different datasets, namely MNIST, FEMNIST, and CIFAR-10, and the results demonstrate that FedRFQ outperforms existing baselines in terms of accuracy when handling non-IID data.
Paper Structure (25 sections, 3 theorems, 34 equations, 10 figures, 3 tables)

This paper contains 25 sections, 3 theorems, 34 equations, 10 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Federated Learning with Non-IID
  4. Prototype-based Federated Learning with Non-IID
  5. Federated Learning with Malicious Attacks
  6. Summary
  7. Preliminaries and Models
  8. Preliminaries
  9. Models and Design Goals
  10. FedRFQ
  11. Overview
  12. Key Steps of FedRFQ
  13. BFT-detect
  14. Analysis
  15. Upper Bound Analysis on the Loss Function with Poisoning Attacks
  16. ...and 10 more sections

Key Result

Theorem 1

(The upper bound) For any client, after each round of communications in FedRFQ, $\mathbb{E}\left[\bm{\ell}_k^{((t+1)E)^\prime}\right]$ is upper bounded by where $\bm{\ell}_k^{(tE)^\prime}$ denotes the local loss function after prototype aggregation in the $t$-th round. Similarly, $\bm{\ell}_k^{((t+1)E)^\prime}$ denotes the local loss function when competing the prototype aggregation in the ($t+1$

Figures (10)

  • Figure 1: An example of prototype failure
  • Figure 2: The model of FedRFQ.
  • Figure 3: The one-round execution of FedRFQ.
  • Figure 4: The pooling operation with SoftPool
  • Figure 5: The security probability with respect to $N$, the total number of servers, under different probability of a server being malicious.
  • ...and 5 more figures

Theorems & Definitions (3)

  • Theorem 1
  • Lemma 1
  • Lemma 2