Towards Communication-Efficient Adversarial Federated Learning for Robust Edge Intelligence

TL;DR

This work tackles robustness and communication efficiency in adversarial federated learning under non-IID data. It introduces PM-AFL and PM-AFL++ that leverage a pre-trained teacher to distill both clean and adversarial knowledge through vanilla and adversarial mixture knowledge distillation, complemented by a global alignment term to mitigate non-IID effects. Experiments on MNIST, CIFAR-10, and CIFAR-100 show that PM-AFL/PM-AFL++ outperform baselines in both accuracy and adversarial robustness while dramatically reducing per-round communication, up to about 73x, 36x, and 23x respectively. The proposed approach enables robust edge intelligence with far lower communication overhead, making AFL more practical for resource-constrained edge networks.

Abstract

Federated learning (FL) has gained significant attention for enabling decentralized training on edge networks without exposing raw data. However, FL models remain susceptible to adversarial attacks and performance degradation in non-IID data settings, thus posing challenges to both robustness and accuracy. This paper aims to achieve communication-efficient adversarial federated learning (AFL) by leveraging a pre-trained model to enhance both robustness and accuracy under adversarial attacks and non-IID challenges in AFL. By leveraging the knowledge from a pre-trained model for both clean and adversarial images, we propose a pre-trained model-guided adversarial federated learning (PM-AFL) framework. This framework integrates vanilla and adversarial mixture knowledge distillation to effectively balance accuracy and robustness while promoting local models to learn from diverse data. Specifically, for clean accuracy, we adopt a dual distillation strategy where the class probabilities of randomly paired images, and their blended versions are aligned between the teacher model and the local models. For adversarial robustness, we employ a similar distillation approach but replace clean samples on the local side with adversarial examples. Moreover, by considering the bias between local and global models, we also incorporate a consistency regularization term to ensure that local adversarial predictions stay aligned with their corresponding global clean ones. These strategies collectively enable local models to absorb diverse knowledge from the teacher model while maintaining close alignment with the global model, thereby mitigating overfitting to local optima and enhancing the generalization of the global model. Experiments demonstrate that the PM-AFL-based framework not only significantly outperforms other methods but also maintains communication efficiency.

Figures (4)

• Figure 1: Illustration of the proposed PM-AFL++ framework. We propose vanilla mixture knowledge distillation ($\mathcal{L}_{VKD}$) in Section \ref{['subsec:vanilla_MKD']}, while adversarial mixture knowledge distillation ($\mathcal{L}_{AKD}$) is presented in Section \ref{['subsec:adversarial_MKD']}. In addition, the alignment between local and global models ($\mathcal{L}_{ALG}$) is discussed in Section \ref{['subsec:alignment_local_global']}.
• Figure 2: Robustness comparison of PM-AFL++ and FedPGD on MNIST under different levels of heterogeneities.
• Figure 3: Accuracy comparison of PM-AFL++ and FedPGD on CIFAR-10 under different levels of heterogeneities.
• Figure 4: Robustness of PM-AFL++ on MNIST, CIFAR-10, and CIFAR-100 under different values of $\rho$. The X-axis represents the ratio of robustness to accuracy, defined as $\rho = \alpha / (1 - \alpha)$. The Y-axis illustrates robustness against a diverse range of attacks. The selected values of $\rho$ for MNIST, CIFAR-10, and CIFAR-100 are 10.0/1.0, 5.0/1.0, and 10.0/1.0, respectively.