Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedSat: A Statistical Aggregation Approach for Class Imbalanced Clients in Federated Learning

Sujit Chowdhury, Raju Halder

TL;DR

FedSat tackles three forms of data heterogeneity in federated learning—label skewness, missing classes, and quantity skewness—by integrating a prediction-sensitive loss and a prioritized-class based weighted aggregation. It formalizes a global objective and employs a two-stage evaluation of client updates via worker sets to compute class-aware statistics, guiding robust, class-aware aggregation. Empirical results on MNIST, CIFAR-10, and CIFAR-100 under LS, LSMC, and LQSMC demonstrate consistent improvements over baselines, with faster convergence and stronger performance on underrepresented classes, including up to substantial gains in extreme non-IID settings. The approach offers a scalable, robust solution for real-world heterogeneous FL and points to future work in privacy-preserving and security-enhanced extensions such as differential privacy, secure aggregation, and blockchain integration.

Abstract

Federated learning (FL) has emerged as a promising paradigm for privacy-preserving distributed machine learning, but faces challenges with heterogeneous data distributions across clients. This paper presents FedSat, a novel FL approach specifically designed to simultaneously handle three forms of data heterogeneity, namely label skewness, missing classes, and quantity skewness, by proposing a prediction-sensitive loss function and a prioritized-class based weighted aggregation scheme. While the prediction-sensitive loss function enhances model performance on minority classes, the prioritized-class based weighted aggregation scheme ensures client contributions are weighted based on both statistical significance and performance on critical classes. Extensive experiments across diverse data-heterogeneity settings demonstrate that FedSat significantly outperforms state-of-the-art baselines, with an average improvement of 1.8% over the second-best method and 19.87% over the weakest-performing baseline. The approach also demonstrates faster convergence compared to existing methods. These results highlight FedSat's effectiveness in addressing the challenges of heterogeneous federated learning and its potential for real-world applications.

FedSat: A Statistical Aggregation Approach for Class Imbalanced Clients in Federated Learning

TL;DR

FedSat tackles three forms of data heterogeneity in federated learning—label skewness, missing classes, and quantity skewness—by integrating a prediction-sensitive loss and a prioritized-class based weighted aggregation. It formalizes a global objective and employs a two-stage evaluation of client updates via worker sets to compute class-aware statistics, guiding robust, class-aware aggregation. Empirical results on MNIST, CIFAR-10, and CIFAR-100 under LS, LSMC, and LQSMC demonstrate consistent improvements over baselines, with faster convergence and stronger performance on underrepresented classes, including up to substantial gains in extreme non-IID settings. The approach offers a scalable, robust solution for real-world heterogeneous FL and points to future work in privacy-preserving and security-enhanced extensions such as differential privacy, secure aggregation, and blockchain integration.

Abstract

Federated learning (FL) has emerged as a promising paradigm for privacy-preserving distributed machine learning, but faces challenges with heterogeneous data distributions across clients. This paper presents FedSat, a novel FL approach specifically designed to simultaneously handle three forms of data heterogeneity, namely label skewness, missing classes, and quantity skewness, by proposing a prediction-sensitive loss function and a prioritized-class based weighted aggregation scheme. While the prediction-sensitive loss function enhances model performance on minority classes, the prioritized-class based weighted aggregation scheme ensures client contributions are weighted based on both statistical significance and performance on critical classes. Extensive experiments across diverse data-heterogeneity settings demonstrate that FedSat significantly outperforms state-of-the-art baselines, with an average improvement of 1.8% over the second-best method and 19.87% over the weakest-performing baseline. The approach also demonstrates faster convergence compared to existing methods. These results highlight FedSat's effectiveness in addressing the challenges of heterogeneous federated learning and its potential for real-world applications.
Paper Structure (30 sections, 4 theorems, 24 equations, 7 figures, 6 tables, 2 algorithms)

This paper contains 30 sections, 4 theorems, 24 equations, 7 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. FedSat: Proposed FL Approach
  5. Learning Objectives
  6. Local Training with Prediction-Sensitive Loss
  7. Learned parameter analysis
  8. Prediction Matrix ($\hat{Z}$)
  9. True Positive Matrix ($TP$)
  10. Target Matrix $(T)$
  11. Weight computation
  12. Global model update
  13. Training process
  14. Convergence Proof
  15. Experimental Evaluation
  16. ...and 15 more sections

Key Result

Lemma 1

Let $\theta_{PS}$ and $\theta_{CE}$ be the parameters obtained after training with prediction-sensitive loss function $\mathcal{L}_{PS}$ and cross-entropy loss functions $\mathcal{L}_{CE}$, respectively. Let both the assumptions assump:1 and assump:2 hold. Then, the expected improvement of the local

Figures (7)

  • Figure 1: FedSat Framework Architecture.
  • Figure 2: Comparison of the highest and lowest client accuracies achieved by FedSat and baseline methods across various settings. The settings are defined by the local learning rate ($\eta_l$), batch size ($\mathbf{B}$), number of global rounds ($\mathbf{R}$), number of classes per client (#C), dataset ($\mathbf{D}$), and model architecture ($\mathbf{M}$).
  • Figure 3: Class-wise accuracy comparison between FedSat and baseline methods on the CIFAR-10 dataset with label skew, where each client is assigned only 2 classes.
  • Figure 4: Top1-accuracy on MNIST Dataset using LeNet5 model architecture.
  • Figure 5: Top-1 accuracy on CIFAR10 dataset using ResNet8 model architecture.
  • ...and 2 more figures

Theorems & Definitions (8)

  • Lemma 1: Improvement Through Prediction-Sensitive Loss
  • Proof 1
  • Lemma 2: Improved Accuracy for Critical Class through Prioritized Class-based Weighted Aggregation Scheme
  • Proof 2
  • Lemma 3: Convergence of Global Model
  • Proof 3
  • Theorem 1: Faster Convergence and Robust Model Generation in FedSat
  • Proof 4