Local K-Similarity Constraint for Federated Learning with Label Noise

TL;DR

The paper tackles federated learning under substantial label noise by introducing a local K-similarity constraint that regresses client representations toward the local SSL neighborhood. A fixed SSL encoder provides pseudo-ground-truth neighborhoods, and an InfoNCE-based regularizer enforces locality in the classifier space, yielding a per-sample objective that combines standard CE with a neighborhood-based penalty. Empirically, the method surpasses state-of-the-art FNLL baselines across computer vision and medical imaging benchmarks, including real-world noisy datasets, and remains effective with different SSL backbones in architecture-agnostic settings. The approach offers practical robustness without requiring shared SSL and classifier architectures or global denoising, making it suitable for diverse federated deployments with noisy client data.

Abstract

Federated learning on clients with noisy labels is a challenging problem, as such clients can infiltrate the global model, impacting the overall generalizability of the system. Existing methods proposed to handle noisy clients assume that a sufficient number of clients with clean labels are available, which can be leveraged to learn a robust global model while dampening the impact of noisy clients. This assumption fails when a high number of heterogeneous clients contain noisy labels, making the existing approaches ineffective. In such scenarios, it is important to locally regularize the clients before communication with the global model, to ensure the global model isn't corrupted by noisy clients. While pre-trained self-supervised models can be effective for local regularization, existing centralized approaches relying on pretrained initialization are impractical in a federated setting due to the potentially large size of these models, which increases communication costs. In that line, we propose a regularization objective for client models that decouples the pre-trained and classification models by enforcing similarity between close data points within the client. We leverage the representation space of a self-supervised pretrained model to evaluate the closeness among examples. This regularization, when applied with the standard objective function for the downstream task in standard noisy federated settings, significantly improves performance, outperforming existing state-of-the-art federated methods in multiple computer vision and medical image classification benchmarks. Unlike other techniques that rely on self-supervised pretrained initialization, our method does not require the pretrained model and classifier backbone to share the same architecture, making it architecture-agnostic.

Figures (14)

• Figure 1: (a) A general federated learning framework with label noise. (b) Illustration of our method: At each client, for a given input image (yellow), we find the $K$ nearest neighbours in the batch (green) in the SSL representation space. The neighbours in the SSL space may be far apart in the classifier space due to the effect of label noise. Thus the representations of $K$-nearest neighbours in the neighbourhood $\mathcal{N}$ are constrained to be closer to the input in the classifier representation space using our local $K$-similarity constraint $L_{CL}$.
• Figure 2: (a) t-SNE plot of features from the MURA dataset, extracted using an ImageNet SSL-pretrained model, showing three distinct classes, each represented by a different color, with visible sub-clusters. (b) Cross-Entropy loss curves for noisy and clean training samples of a randomly selected client in the (0.7, 0.5) noise setting trained with FedAvg and our method. (c) Magnitude of the gradient for clean and noisy samples across 10 high-noise clients in the (0.7, 0.5) noise setting.
• Figure 3: Comparison of our local regularization approach with FedLN-AKD visualizing 3 classes (blue, grey, and orange). Please zoom in for a better view. Colors represent ground truth labels while shapes with red borders represent samples with wrongly assigned labels-assigned labels represented by shape. Solid and hollow shapes of the same color refer to elements of different sub-class clusters belonging to the same parent class. Thin red arrows illustrate the tendency of CE to bring embeddings with the same assigned labels close. Thin green curves in FedLN-AKD illustrate the regression of embeddings according to the SSL space.
• Figure 4: Best Accuracy on CIFAR-10 at various values of $K$ at $\mathcal{T}$ set to 0.3 (left) and various values of $\mathcal{T}$ at $K$ set to 4 (right).
• Figure 5: Best Accuracy and convergence rate reported at various values of $\lambda$ for $k=5$ for CIFAR-10.