Learning Locally, Revising Globally: Global Reviser for Federated Learning with Noisy Labels

TL;DR

This work addresses the challenge of label-noise in federated learning by revealing that the global FL model memorizes noisy labels slowly, enabling a server-guided approach. FedGR combines three modules—sniffing-then-refining to identify and correct noisy labels, global revised EMA distillation to leverage shared knowledge for client models, and global representation regularization to prevent overfitting to noise. Empirical results across CIFAR-10/100 and Clothing1M demonstrate that FedGR consistently outperforms state-of-the-art baselines under diverse F-LNL scenarios, even approaching clean-data performance when some clients are clean. The approach offers a privacy-preserving, computation-efficient path to robust Federated Learning in real-world, noisy-label settings with heterogeneous data distributions.

Abstract

The success of most federated learning (FL) methods heavily depends on label quality, which is often inaccessible in real-world scenarios, such as medicine, leading to the federated label-noise (F-LN) problem. In this study, we observe that the global model of FL memorizes the noisy labels slowly. Based on the observations, we propose a novel approach dubbed Global Reviser for Federated Learning with Noisy Labels (FedGR) to enhance the label-noise robustness of FL. In brief, FedGR employs three novel modules to achieve noisy label sniffing and refining, local knowledge revising, and local model regularization. Specifically, the global model is adopted to infer local data proxies for global sample selection and refine incorrect labels. To maximize the utilization of local knowledge, we leverage the global model to revise the local exponential moving average (EMA) model of each client and distill it into the clients' models. Additionally, we introduce a global-to-local representation regularization to mitigate the overfitting of noisy labels. Extensive experiments on three F-LNL benchmarks against seven baseline methods demonstrate the effectiveness of the proposed FedGR.

Figures (17)

• Figure 1: Observation: The global model of FL memorizes label-noise data slowly and learns underlying correct knowledge. According to (a), the global model of FL memorizes no more than 30% label-noise samples over the whole training, and as a contrast the typical centrally trained model memorizes over 80% label-noise data. Following (b), the global model does not encounter the test performance drop like the centrally trained model and achieves promising test performance. Experiments are conducted on CIFAR-10 with client sample ratio 0.2 and 10 clients, each randomly assigned symmetric or asymmetric noisy labels. The overall label-noise ratio denotes the proportion of noisy samples over the whole dataset. See the appendix for details.
• Figure 2: Illustration of FedGR, which effectively leverages the characteristics of the global model to achieve robust F-LNL. It performs centralized sieving and label refining to rectify the label-noise data in sniffing-then-refining § \ref{['sec:sr']}. To leverage all the local knowledge, we distill the revised local EMA model's logits to the client's model via global revised EMA distillation § \ref{['sec:b']}. To further regularize the local learning, a global-to-local representation distillation is adopted, namely global representation regularization § \ref{['sec:r']}.
• Figure 3: Local EMA model updating process on client $k$ at round $t$, where $m_k$ denotes the number of the corresponding local training steps.
• Figure 4: The (a) and (b) are the pearson coefficient analysis of the label-noise sniffing under different data partitions. The (c) and (d) are the clients' F-score distributions of different methods under different data partitions. The F-LNL setup: CIFAR-10, Sym, $\phi=1.0$, and $\mathcal{U}(0.5,1.0)$.
• Figure 5: Memorization effect observation experimental results with FL client sample ratio 0.2.