Towards Fair, Robust and Efficient Client Contribution Evaluation in Federated Learning

TL;DR

This work tackles the challenge of valuing per-client contributions in Federated Learning under non-iid data and in the presence of Byzantine attackers, without relying on a validation dataset. It introduces FRECA, a framework that uses FedTruth to estimate the ground-truth global update, balancing client contributions while filtering malicious updates via a Byzantine-resilient aggregation process. FRECA defines two metrics—Client Performance (AW) and Net Contribution—to quantify reliability and the actual impact on the global model, with distances and regulation functions governing the attribution. Empirical results on MNIST, CIFAR-10, and FashionMNIST demonstrate that FRECA provides accurate, robust, and efficient contribution estimates, improves attacker detection, and significantly reduces computation time compared to Shapley-value-based methods.

Abstract

The performance of clients in Federated Learning (FL) can vary due to various reasons. Assessing the contributions of each client is crucial for client selection and compensation. It is challenging because clients often have non-independent and identically distributed (non-iid) data, leading to potentially noisy or divergent updates. The risk of malicious clients amplifies the challenge especially when there's no access to clients' local data or a benchmark root dataset. In this paper, we introduce a novel method called Fair, Robust, and Efficient Client Assessment (FRECA) for quantifying client contributions in FL. FRECA employs a framework called FedTruth to estimate the global model's ground truth update, balancing contributions from all clients while filtering out impacts from malicious ones. This approach is robust against Byzantine attacks and incorporates a Byzantine-resilient aggregation algorithm. FRECA is also efficient, as it operates solely on local model updates and requires no validation operations or datasets. Our experimental results show that FRECA can accurately and efficiently quantify client contributions in a robust manner.

Figures (8)

• Figure 1: Gap Distance between all model updates and the estimated global model update
• Figure 2: Contribution for Case 1: # labels for Client 0-7: 1, 2, 3, 4, 6, 8, 9, 10
• Figure 3: Contribution for Case 2: # labels for Client 0-7: 1, 1, 1, 1, 1, 1, 2, 2
• Figure 4: Contribution for Case 3: iid, 1 attacker
• Figure 5: Contribution for Case 4: iid, 2 attackers