Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clients

TL;DR

RAHFL tackles robust heterogeneous Federated Learning by integrating a Diversity-enhanced supervised Contrastive Learning (DCL) module for resilient local representation learning with an Asymmetric Heterogeneous Federated Learning (AsymHFL) mechanism for selective knowledge transfer via a public dataset. It combines random mixed augmentation with Jensen-Shannon consistency and a KL-divergence-based collaboration framework using a dynamically updated knowledge-transfer matrix to prevent propagation of corrupted feedback. Across CIFAR-10-C and related datasets, RAHFL consistently outperforms state-of-the-art heterogeneous FL methods under varying corruption levels, in both heterogeneous and homogeneous model settings, demonstrating strong robustness and practical applicability. The approach offers a principled pathway to robust, scalable FL in real-world deployments, with public-code availability enabling reproducibility and broader adoption.

Abstract

This paper studies a challenging robust federated learning task with model heterogeneous and data corrupted clients, where the clients have different local model structures. Data corruption is unavoidable due to factors such as random noise, compression artifacts, or environmental conditions in real-world deployment, drastically crippling the entire federated system. To address these issues, this paper introduces a novel Robust Asymmetric Heterogeneous Federated Learning (RAHFL) framework. We propose a Diversity-enhanced supervised Contrastive Learning technique to enhance the resilience and adaptability of local models on various data corruption patterns. Its basic idea is to utilize complex augmented samples obtained by the mixed-data augmentation strategy for supervised contrastive learning, thereby enhancing the ability of the model to learn robust and diverse feature representations. Furthermore, we design an Asymmetric Heterogeneous Federated Learning strategy to resist corrupt feedback from external clients. The strategy allows clients to perform selective one-way learning during collaborative learning phase, enabling clients to refrain from incorporating lower-quality information from less robust or underperforming collaborators. Extensive experimental results demonstrate the effectiveness and robustness of our approach in diverse, challenging federated learning environments. Our code and models are public available at https://github.com/FangXiuwen/RAHFL.

Figures (6)

• Figure 1: Illustration of heterogeneous FL with data corruption, where the clients may possess different model structures and corrupted private datasets.
• Figure 2: Model performance on clean vs. corrupted test data. We observe that corrupted data is more susceptible to misprediction than clean data. RAHFL outperforms vanilla models on corrupted data while maintaining accuracy on clean data.
• Figure 3: Overview of the RAHFL framework. The RAHFL framework consists of a local learning phase and a collaborative learning phase. In the local learning phase, random mixed augmentation and DCL are introduced to mitigate the negative impact of local data corruption (Sec.\ref{['sec:localupdate']}). In the collaborative learning phase, clients utilize the AsymHFL strategy for asymmetric and efficient knowledge transfer, thereby avoiding the integration of low-quality information (Sec.\ref{['sec:colupdate']}). The diagram shows the differences between traditional HFL and the new AsymHFL approach.
• Figure 4: Comparison of test accuracy during the FL process across homogeneous models, when the corruption rate $\xi=0.5$. The x-axis represents communication epochs, and the y-axis denotes test accuracy.
• Figure R1: Illustration of the number of samples per class in each client when the Dirichlet distribution beta value is 1.0. The size of each point represents the quantity of samples for each class in the respective client.