Federated Learning with Instance-Dependent Noisy Label
Lei Wang, Jieming Bian, Jie Xu
TL;DR
Instance-dependent label noise presents a critical challenge for federated learning due to small, heterogeneous client datasets. FedBeat adopts an instance-dependent noise transition matrix $T(\mathbf{x})$ and a three-step pipeline—Federated Data Extraction with Bayesian ensemble pseudo-labeling, Federated Transition Matrix Estimation, and Federated Classifier Correction—to construct a globally consistent classifier. The approach formalizes the IDN setting, trains an IDNTM estimation network, and demonstrates substantial accuracy gains on CIFAR-10 and SVHN compared with strong FL baselines. This work enables robust federated learning under realistic noisy labeling conditions by explicitly modeling label corruption and correcting predictions in a distributed, privacy-preserving manner.
Abstract
Federated learning (FL) with noisy labels poses a significant challenge. Existing methods designed for handling noisy labels in centralized learning tend to lose their effectiveness in the FL setting, mainly due to the small dataset size and the heterogeneity of client data. While some attempts have been made to tackle FL with noisy labels, they primarily focused on scenarios involving class-conditional noise. In this paper, we study the more challenging and practical issue of instance-dependent noise (IDN) in FL. We introduce a novel algorithm called FedBeat (Federated Learning with Bayesian Ensemble-Assisted Transition Matrix Estimation). FedBeat aims to build a global statistically consistent classifier using the IDN transition matrix (IDNTM), which encompasses three synergistic steps: (1) A federated data extraction step that constructs a weak global model and extracts high-confidence data using a Bayesian model ensemble method. (2) A federated transition matrix estimation step in which clients collaboratively train an IDNTM estimation network based on the extracted data. (3) A federated classifier correction step that enhances the global model's performance by training it using a loss function tailored for noisy labels, leveraging the IDNTM. Experiments conducted on CIFAR-10 and SVHN verify that the proposed method significantly outperforms state-of-the-art methods.