Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning Client Pruning for Noisy Labels

Mahdi Morafah, Hojin Chang, Chen Chen, Bill Lin

TL;DR

Empirical evaluation demonstrates ClipFL’s efficacy across diverse datasets and noise levels, achieving accurate noisy client identification, superior performance, faster convergence, and reduced communication costs compared to state-of-the-art FL methods.

Abstract

Federated Learning (FL) enables collaborative model training across decentralized edge devices while preserving data privacy. However, existing FL methods often assume clean annotated datasets, impractical for resource-constrained edge devices. In reality, noisy labels are prevalent, posing significant challenges to FL performance. Prior approaches attempt label correction and robust training techniques but exhibit limited efficacy, particularly under high noise levels. This paper introduces ClipFL (Federated Learning Client Pruning), a novel framework addressing noisy labels from a fresh perspective. ClipFL identifies and excludes noisy clients based on their performance on a clean validation dataset, tracked using a Noise Candidacy Score (NCS). The framework comprises three phases: pre-client pruning to identify potential noisy clients and calculate their NCS, client pruning to exclude a percentage of clients with the highest NCS, and post-client pruning for fine-tuning the global model with standard FL on clean clients. Empirical evaluation demonstrates ClipFL's efficacy across diverse datasets and noise levels, achieving accurate noisy client identification, superior performance, faster convergence, and reduced communication costs compared to state-of-the-art FL methods. Our code is available at https://github.com/MMorafah/ClipFL.

Federated Learning Client Pruning for Noisy Labels

TL;DR

Empirical evaluation demonstrates ClipFL’s efficacy across diverse datasets and noise levels, achieving accurate noisy client identification, superior performance, faster convergence, and reduced communication costs compared to state-of-the-art FL methods.

Abstract

Federated Learning (FL) enables collaborative model training across decentralized edge devices while preserving data privacy. However, existing FL methods often assume clean annotated datasets, impractical for resource-constrained edge devices. In reality, noisy labels are prevalent, posing significant challenges to FL performance. Prior approaches attempt label correction and robust training techniques but exhibit limited efficacy, particularly under high noise levels. This paper introduces ClipFL (Federated Learning Client Pruning), a novel framework addressing noisy labels from a fresh perspective. ClipFL identifies and excludes noisy clients based on their performance on a clean validation dataset, tracked using a Noise Candidacy Score (NCS). The framework comprises three phases: pre-client pruning to identify potential noisy clients and calculate their NCS, client pruning to exclude a percentage of clients with the highest NCS, and post-client pruning for fine-tuning the global model with standard FL on clean clients. Empirical evaluation demonstrates ClipFL's efficacy across diverse datasets and noise levels, achieving accurate noisy client identification, superior performance, faster convergence, and reduced communication costs compared to state-of-the-art FL methods. Our code is available at https://github.com/MMorafah/ClipFL.

Paper Structure

This paper contains 22 sections, 8 equations, 15 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Background and Problem Formulation
  4. Background
  5. Problem Formulation
  6. Proposed Method
  7. Overview
  8. Phase I: Pre-Client Pruning
  9. Phase II: Client Pruning
  10. Phase III: Post-Client Pruning
  11. Experiments
  12. Experiment Settings
  13. RQ1: Comparison with SOTA FL optimizers
  14. RQ2: Comparison with SOTA FL methods that deal with noisy labels
  15. RQ3: Impact of ClipFL's hyper-parameters
  16. ...and 7 more sections

Figures (15)

  • Figure 1: Prior works typically involve incorporating noisy clients into FL and addressing their negative impact through methods such as correcting noisy labels, re-weighting noisy clients, or designing robust local training methods.
  • Figure 2: In contrast to prior works, ClipFL introduces the Noise Candidacy Score (NCS) for each client, enabling robust identification of noisy clients from the FL process.
  • Figure 4: Symmetric noise transition matrix
  • Figure 5: $\mu=0.5$
  • Figure 6: $\mu=0.8$
  • ...and 10 more figures