FLGuard: Byzantine-Robust Federated Learning via Ensemble of Contrastive Models

TL;DR

This paper tackles poisoning in federated learning by introducing FLGuard, a Byzantine-robust defense that does not rely on prior knowledge such as the number of malicious clients or auxiliary data. It uses an ensemble of SimCLR-based contrastive models to learn robust representations of local updates, coupled with PCA and clustering to filter out malicious clients before aggregation. The method updates the contrastive models every k FL rounds to adapt to drift and achieves state-of-the-art robustness across MPAs and DPAs, especially under non-IID data, while preserving fidelity and maintaining low overhead. The work suggests future improvements like more negative pairs and hybrid approaches that combine dimension-wise and vector-wise defenses for even stronger protection.

Abstract

Federated Learning (FL) thrives in training a global model with numerous clients by only sharing the parameters of their local models trained with their private training datasets. Therefore, without revealing the private dataset, the clients can obtain a deep learning (DL) model with high performance. However, recent research proposed poisoning attacks that cause a catastrophic loss in the accuracy of the global model when adversaries, posed as benign clients, are present in a group of clients. Therefore, recent studies suggested byzantine-robust FL methods that allow the server to train an accurate global model even with the adversaries present in the system. However, many existing methods require the knowledge of the number of malicious clients or the auxiliary (clean) dataset or the effectiveness reportedly decreased hugely when the private dataset was non-independently and identically distributed (non-IID). In this work, we propose FLGuard, a novel byzantine-robust FL method that detects malicious clients and discards malicious local updates by utilizing the contrastive learning technique, which showed a tremendous improvement as a self-supervised learning method. With contrastive models, we design FLGuard as an ensemble scheme to maximize the defensive capability. We evaluate FLGuard extensively under various poisoning attacks and compare the accuracy of the global model with existing byzantine-robust FL methods. FLGuard outperforms the state-of-the-art defense methods in most cases and shows drastic improvement, especially in non-IID settings. https://github.com/201younghanlee/FLGuard

Figures (5)

• Figure 1: Overview of byzantine-robust FL. Steps described in asterisk (*) denote the design novelty of our FLGuard
• Figure 2: Flowchart of FLGuard Process (Preprocessing, Training Contrastive Models, and Filtering Malicious Clients). We note that the training phase only occurs every k FL rounds to generate new contrastive models
• Figure 3: Illustration of Contrastive Learning including Augmentation. $f(\cdot)$ and $g(\cdot)$ represent the contrastive models (encoders) and the projection heads
• Figure 4: Global Model Accuracy vs. FL rounds for FLGuard with various MPAs and FedAvg without attacks
• Figure 5: Impact of Fraction of Malicious Clients (FEMNIST, (a)-(d)) and Non-IID Degree (CIFAR10, (e)-(h)) on Accuracy against MPAs (STAT-OPT (Tmean), Min-Sum (sgn), DYN-OPT (Tmean), LIE (sgn)) and Threat Models (T1-T4)