Contribution Evaluation of Heterogeneous Participants in Federated Learning via Prototypical Representations

TL;DR

This paper tackles the challenge of evaluating contributions of heterogeneous participants in federated learning without relying on auxiliary test data. It introduces FLCE, a representation-based framework that uses class contribution momentum, combining class contribution mass and velocity across individual, relative, and holistic perspectives. Through extensive experiments on CIFAR-10/100 and EuroSAT, FLCE demonstrates strong fidelity, effectiveness, and efficiency, outperforming many baselines and approaching Shapley-value methods in a fraction of the computational cost. The approach also supports class-level contribution analysis and shows robustness to data noise and statistical heterogeneity, offering a practical and scalable solution for fair and informative participant evaluation in FL.

Abstract

Contribution evaluation in federated learning (FL) has become a pivotal research area due to its applicability across various domains, such as detecting low-quality datasets, enhancing model robustness, and designing incentive mechanisms. Existing contribution evaluation methods, which primarily rely on data volume, model similarity, and auxiliary test datasets, have shown success in diverse scenarios. However, their effectiveness often diminishes due to the heterogeneity of data distributions, presenting a significant challenge to their applicability. In response, this paper explores contribution evaluation in FL from an entirely new perspective of representation. In this work, we propose a new method for the contribution evaluation of heterogeneous participants in federated learning (FLCE), which introduces a novel indicator \emph{class contribution momentum} to conduct refined contribution evaluation. Our core idea is the construction and application of the class contribution momentum indicator from individual, relative, and holistic perspectives, thereby achieving an effective and efficient contribution evaluation of heterogeneous participants without relying on an auxiliary test dataset. Extensive experimental results demonstrate the superiority of our method in terms of fidelity, effectiveness, efficiency, and heterogeneity across various scenarios.

Figures (9)

• Figure 1: Framework of the proposed FLCE for contribution evaluation of heterogeneous participants in federated learning.
• Figure 2: Effectiveness of various methods in contribution evaluation of heterogeneous participants in FL.
• Figure 3: KL divergence between contribution evaluation results and data quality based on class diversity.
• Figure 4: Differences between participant contributions calculated by each algorithm and canonical Shapley Value.
• Figure 5: Comparison of class contribution weights obtained by different methods and ground truth weight.