Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedEFC: Federated Learning Using Enhanced Forward Correction Against Noisy Labels

Seunghun Yu, Jin-Hyun Ahn, Joonhyuk Kang

TL;DR

This work develops an effective loss correction tailored to the unique challenges of FL, including data heterogeneity and decentralized training, and provides a theoretical analysis, leveraging the composite proper loss property, to demonstrate that the FL objective function under noisy label distributions can be aligned with the clean label distribution.

Abstract

Federated Learning (FL) is a powerful framework for privacy-preserving distributed learning. It enables multiple clients to collaboratively train a global model without sharing raw data. However, handling noisy labels in FL remains a major challenge due to heterogeneous data distributions and communication constraints, which can severely degrade model performance. To address this issue, we propose FedEFC, a novel method designed to tackle the impact of noisy labels in FL. FedEFC mitigates this issue through two key techniques: (1) prestopping, which prevents overfitting to mislabeled data by dynamically halting training at an optimal point, and (2) loss correction, which adjusts model updates to account for label noise. In particular, we develop an effective loss correction tailored to the unique challenges of FL, including data heterogeneity and decentralized training. Furthermore, we provide a theoretical analysis, leveraging the composite proper loss property, to demonstrate that the FL objective function under noisy label distributions can be aligned with the clean label distribution. Extensive experimental results validate the effectiveness of our approach, showing that it consistently outperforms existing FL techniques in mitigating the impact of noisy labels, particularly under heterogeneous data settings (e.g., achieving up to 41.64% relative performance improvement over the existing loss correction method).

FedEFC: Federated Learning Using Enhanced Forward Correction Against Noisy Labels

TL;DR

This work develops an effective loss correction tailored to the unique challenges of FL, including data heterogeneity and decentralized training, and provides a theoretical analysis, leveraging the composite proper loss property, to demonstrate that the FL objective function under noisy label distributions can be aligned with the clean label distribution.

Abstract

Federated Learning (FL) is a powerful framework for privacy-preserving distributed learning. It enables multiple clients to collaboratively train a global model without sharing raw data. However, handling noisy labels in FL remains a major challenge due to heterogeneous data distributions and communication constraints, which can severely degrade model performance. To address this issue, we propose FedEFC, a novel method designed to tackle the impact of noisy labels in FL. FedEFC mitigates this issue through two key techniques: (1) prestopping, which prevents overfitting to mislabeled data by dynamically halting training at an optimal point, and (2) loss correction, which adjusts model updates to account for label noise. In particular, we develop an effective loss correction tailored to the unique challenges of FL, including data heterogeneity and decentralized training. Furthermore, we provide a theoretical analysis, leveraging the composite proper loss property, to demonstrate that the FL objective function under noisy label distributions can be aligned with the clean label distribution. Extensive experimental results validate the effectiveness of our approach, showing that it consistently outperforms existing FL techniques in mitigating the impact of noisy labels, particularly under heterogeneous data settings (e.g., achieving up to 41.64% relative performance improvement over the existing loss correction method).

Paper Structure

This paper contains 17 sections, 1 theorem, 17 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Data Allocation
  5. Noise Generation
  6. Proposed Method
  7. Finding the Prestopping Point
  8. Applying Loss Correction
  9. Generating the Noise Transition Matrix
  10. Applying Loss Correction
  11. Objective Function Analysis in Noisy Labels
  12. Experiments
  13. Experiment Setup
  14. Implementation Details
  15. Comparison with Baselines
  16. ...and 2 more sections

Key Result

Theorem 1

Assume each matrix $Q^k_{\tilde{y}|y}$ generated by client $k$ is non-singular, and approximately equal to the true transition matrix $T^k_{\tilde{y}|y}$ whose $(i,j)$th entries are true conditional probabilities $p(\tilde{y}=i|y=j)$. A composite loss incorporating $Q^{k}_{{\tilde{y}}|{y}}$ is given Then, the aggregated objective at the minimizer for clean data is approximately equal to that at th

Figures (3)

  • Figure 1: Overview of FedEFC framework. The scheme consists of two phases: (1) determining the prestopping point and (2) refining the loss correction. In Phase 1, the centralized server tracks client training accuracies to identify the prestopping point where model parameters are near-optimal. In Phase 2, each client updates its loss function using enhanced forward correction, guiding global model parameters toward their optimal configuration in the clean data space.
  • Figure 2: Left: Test accuracy and estimated accuracy used to determine the prestopping point. Right: Cosine similarity between the real noise matrix and the noise matrices estimated by the pretrained model and the global model in training.
  • Figure 3: Example of generating the count matrix $C_{\tilde{y}|y}$. The figure illustrates the process for three labels—cat, dog, and hen—where “labeled” indicates data annotated with the observed label, and the true label is determined based on the threshold $\tau_\text{label}$.

Theorems & Definitions (1)

  • Theorem 1