FedSECA: Sign Election and Coordinate-wise Aggregation of Gradients for Byzantine Tolerant Federated Learning

TL;DR

The proposed FedSECA method for robust sign election and coordinate-wise aggregation of gradients in FL that is less susceptible to malicious updates by an omniscient attacker is compared.

Abstract

One of the most common defense strategies against Byzantine clients in federated learning (FL) is to employ a robust aggregator mechanism that makes the training more resilient. While many existing Byzantine robust aggregators provide theoretical convergence guarantees and are empirically effective against certain categories of attacks, we observe that certain high-strength attacks can subvert the robust aggregator and collapse the training. To overcome this limitation, we propose a method called FedSECA for robust Sign Election and Coordinate-wise Aggregation of gradients in FL that is less susceptible to malicious updates by an omniscient attacker. The proposed method has two main components. The Concordance Ratio Induced Sign Election(CRISE) module determines the consensus direction (elected sign) for each individual parameter gradient through a weighted voting strategy. The client weights are assigned based on a novel metric called concordance ratio, which quantifies the degree of sign agreement between the client gradient updates. Based on the elected sign, a Robust Coordinate-wise Aggregation(RoCA) strategy is employed, where variance-reduced sparse gradients are aggregated only if they are in alignment with the corresponding elected sign. We compare our proposed FedSECA method against 10 robust aggregators under 7 Byzantine attacks on 3 datasets and architectures. The results show that existing robust aggregators fail for at least some attacks, while FedSECA exhibits better robustness. Code - https://github.com/JosephGeoBenjamin/FedSECA-ByzantineTolerance

Figures (4)

• Figure A.1: We split EuroSATdata-helber2019eurosat into 7 clients based on subregion grouping according to "The World Factbook". We use the geolocation information in GeoTIFF files to identify the countries of origin, based on which we assign images to specific clients.
• Figure A.2: We split dataset into 5 clients with class imbalance.
• Figure A.3: We use exact clients split in FedISICdata-ogier2022flamby, which splits dataset based on devices and hospital of origin.
• Figure C.4: Gradients Concordance Ratio ($\rho$): Evolution of $\rho$ values for each client throughout the training computed in CRISE. Green lines correspond to honest clients, Red lines correspond to malicious clients. Client-level Concordance Ratio is used solely for voting weightage; once the optimal gradient signs are chosen for each parameter, the gradient filtering follows an egalitarian approach only considering the alignment of the signs but not the $\rho$. If we filter at the client level solely based on $\rho$, there is a high chance that an honest client might be ignored in its entirety and malicious clients might be included, especially as seen in the ALIE attack. Although ALIE evades the detection while computing $\rho$, it will be filtered out in the sparsification step, criticality of the sparsification step in neutralizing ALIE is also shown in Paper-Tab.1. Although it can be seen that some honest clients might have lower weightage, especially later in training, this has little effect on convergence.