FedSAC: Dynamic Submodel Allocation for Collaborative Fairness in Federated Learning

TL;DR

FedSAC tackles collaborative fairness in federated learning by introducing bounded collaborative fairness (BCF) and a submodel-based allocation strategy. It combines a Submodel Allocation Module that assigns high-performance submodels to high-contributing clients with a Dynamic Aggregation Module that treats low-frequency neurons equitably during aggregation, all under theoretical fairness guarantees and convergence analysis. Empirical results on CIFAR10, SVHN, and Fashion-MNIST show FedSAC outperforms baselines in both fairness (via Pearson correlation-based metrics) and accuracy, with ablations confirming the two core components are essential. The approach reduces communication overhead, preserves local model consistency, and provides a practical path toward broader client participation in FL.

Abstract

Collaborative fairness stands as an essential element in federated learning to encourage client participation by equitably distributing rewards based on individual contributions. Existing methods primarily focus on adjusting gradient allocations among clients to achieve collaborative fairness. However, they frequently overlook crucial factors such as maintaining consistency across local models and catering to the diverse requirements of high-contributing clients. This oversight inevitably decreases both fairness and model accuracy in practice. To address these issues, we propose FedSAC, a novel Federated learning framework with dynamic Submodel Allocation for Collaborative fairness, backed by a theoretical convergence guarantee. First, we present the concept of "bounded collaborative fairness (BCF)", which ensures fairness by tailoring rewards to individual clients based on their contributions. Second, to implement the BCF, we design a submodel allocation module with a theoretical guarantee of fairness. This module incentivizes high-contributing clients with high-performance submodels containing a diverse range of crucial neurons, thereby preserving consistency across local models. Third, we further develop a dynamic aggregation module to adaptively aggregate submodels, ensuring the equitable treatment of low-frequency neurons and consequently enhancing overall model accuracy. Extensive experiments conducted on three public benchmarks demonstrate that FedSAC outperforms all baseline methods in both fairness and model accuracy. We see this work as a significant step towards incentivizing broader client participation in federated learning. The source code is available at https://github.com/wangzihuixmu/FedSAC.

Figures (6)

• Figure 1: Problem illustration of collaborative fairness in FL.(a) Conventional gradients-based methods will result in poor fairness and model accuracy. For example, it is unfair that obtained models of $Client_2$ and $Client_1$ are equivalent neglecting the inferior contribution ($c_1$) of $Client_1$. Plus, the inconsistency in local models results in that obtained models of $Client_2$ and $Client_3$ are worse than expected ($\theta_i^*$). (b) Our proposed FedSAC allocates sufficient submodels to each client by ensuring a comprehensive balance between fairness and model accuracy. For example, FedSAC ensures that obtained models of all clients (i.e., $Client_1$, $Client_2$ and $Client_3$) are in accordance with their contributions respectively. In addition, FedSAC guarantees the alignment of ③ and ④ of $Client_2$ during the training process, thereby enabling all three clients to obtain their expected models ($\theta_i^*$).
• Figure 2: The overall framework of FedSAC that achieves bounded collaborative fairness by maintaining consistency across local models. FedSAC consists of two module: 1) submodel allocation module conducts neuron importance evaluation and submodel construction to reward high-contributing clients with high-performance submodels, thus ensuring consistency in local models; 2) dynamic aggregation module treats those low-frequency neurons equally, which further refines the performance of the global model.
• Figure 3: Comparison results of overall performance to achieve bounded collaborative fairness with state-of-the-art methods in CIFAR10 (left), SVHN (middle), and Fashion MNIST (right). (The closer the point is to the upper-right corner, the better the performance.)
• Figure 4: Comparison results of test accuracy using the data partition of DIR (1.0) with state-of-the-art methods in CIFAR10 (up) and SVHN (down). Results of other scenes are in Appendix D.
• Figure 5: The test accuracy achieved by clients during training for CIFAR10 (left) and SVHN (right) in each round, under the setting of POW and CLA.

Theorems & Definitions (4)

• Definition 1: Bounded Collaborative Fairness
• Theorem 1: Fairness in Training Loss
• Theorem 2: Asymptotic convergence
• lemma 1