Redefining Contributions: Shapley-Driven Federated Learning

TL;DR

This paper tackles fairness and convergence in cross-silo Federated Learning under heterogeneous data by introducing Class-Specific Shapley Values (CSSV) to measure participant contributions at the class level. It presents ShapFed, a method that uses cosine-based last-layer gradients to approximate class-wise Shapley contributions without sharing raw data, and ShapFed-WA, a weighted aggregation scheme that improves global and per-client performance. Personalization is then achieved by tailoring updates to each participant based on their contribution, promoting collaborative fairness. Empirical results on CIFAR-10, Chest X-Ray, and Fed-ISIC2019 demonstrate gains in utility, efficiency, and fairness, with code available for replication.

Abstract

Federated learning (FL) has emerged as a pivotal approach in machine learning, enabling multiple participants to collaboratively train a global model without sharing raw data. While FL finds applications in various domains such as healthcare and finance, it is challenging to ensure global model convergence when participants do not contribute equally and/or honestly. To overcome this challenge, principled mechanisms are required to evaluate the contributions made by individual participants in the FL setting. Existing solutions for contribution assessment rely on general accuracy evaluation, often failing to capture nuanced dynamics and class-specific influences. This paper proposes a novel contribution assessment method called ShapFed for fine-grained evaluation of participant contributions in FL. Our approach uses Shapley values from cooperative game theory to provide a granular understanding of class-specific influences. Based on ShapFed, we introduce a weighted aggregation method called ShapFed-WA, which outperforms conventional federated averaging, especially in class-imbalanced scenarios. Personalizing participant updates based on their contributions further enhances collaborative fairness by delivering differentiated models commensurate with the participant contributions. Experiments on CIFAR-10, Chest X-Ray, and Fed-ISIC2019 datasets demonstrate the effectiveness of our approach in improving utility, efficiency, and fairness in FL systems. The code can be found at https://github.com/tnurbek/shapfed.

Figures (12)

• Figure 1: Overview of our proposedShapFedalgorithm: Each participant $i$ transmits their locally computed iterates $w_i$ to the server. The server then, (i) computes class-specific Shapley values (CSSVs) using the last layer parameters (gradients) $\hat{w}$ (as illustrated in Figure \ref{['fig: network-illustration']}), (ii) aggregates the weights by employing normalized contribution assessment values $\tilde{\gamma}_i$ for each participant $i$, and (iii) broadcasts the personalized weights $\bar{w}_i$ to each participant, using their individual, not-normalized contribution values $\gamma_i$.
• Figure 2: Illustration of the specific weight segments utilized for evaluating class-wise contributions and participant heterogeneity.
• Figure 3: Experimental demonstration comparing performance of utility functions $\nu$ on a synthetic dataset. The experiment contrasts the true Shapley value approach, using validation accuracy across all coalitions, with our proposed approximation method (Section \ref{['sec: ca-approx']}).
• Figure 4: Weighted Aggregation: The optimal weights $w_s^{\star}$ are derived using Equation \ref{['eq: weighted-aggregation']}, while $w_s$ represents the result of applying equal weights (FedAvg). Personalization: Rather than distributing a uniform global model to all users, we provide personalized weights $\bar{w}_i$, which are $\gamma_i$ combinations of individual user weights $w_i$ and the optimally aggregated weight $w_s^{\star}$.
• Figure 5: Comparison of our proposed contribution assessment algorithm (CSSV) with CGSV and true Shapley value computations using ResNet-34 architecture on Chest X-Ray dataset.

Theorems & Definitions (1)

• Example 1: Identification of Class-specific Data Heterogeneity