FeDABoost: Fairness Aware Federated Learning with Adaptive Boosting

TL;DR

FeDABoost tackles fairness in non-IID federated learning by combining a dynamic, AdaBoost-inspired client weighting with an adaptive boosting of underperforming clients via focal loss. The method treats clients as weak learners, assigning positive importance to those with better local performance while boosting hard examples through a growing gamma parameter, controlled by a learning-rate factor \\eta. Empirical results on MNIST, FEMNIST, and CIFAR10 show improved fairness (lower Var(\\varphi)) and competitive or superior accuracy (F1) compared to FedAvg and Ditto, with notable gains under limited participation and non-IID partitions. The work demonstrates that balancing aggregation strength with targeted local emphasis can yield more uniform and robust global models, offering practical impact for cross-silo FL where fairness and efficiency are critical. Future directions include exploring alternative boosting strategies and adaptive scheduling to further stabilize performance across diverse settings.

Abstract

This work focuses on improving the performance and fairness of Federated Learning (FL) in non IID settings by enhancing model aggregation and boosting the training of underperforming clients. We propose FeDABoost, a novel FL framework that integrates a dynamic boosting mechanism and an adaptive gradient aggregation strategy. Inspired by the weighting mechanism of the Multiclass AdaBoost (SAMME) algorithm, our aggregation method assigns higher weights to clients with lower local error rates, thereby promoting more reliable contributions to the global model. In parallel, FeDABoost dynamically boosts underperforming clients by adjusting the focal loss focusing parameter, emphasizing hard to classify examples during local training. We have evaluated FeDABoost on three benchmark datasets MNIST, FEMNIST, and CIFAR10, and compared its performance with those of FedAvg and Ditto. The results show that FeDABoost achieves improved fairness and competitive performance.

Figures (3)

• Figure 1: Ex.1: MNIST. All models are trained with SGD (learning rate = $1 \times 10^{-3}$, batch size = 32, weight decay = $1 \times 10^{-3}$, local epochs = 5); FeDABoost$\mathcal{\eta}=0.01$ and error threshold = 0.3. Total clients: 264.
• Figure 2: Ex.2: FEMNIST. All models are trained for 5 local epochs. The global models of FedAvg, FedABoost-1, and Ditto use SGD (learning rate = $10^{-3}$, batch size = 64, weight decay = $5 \times 10^{-4}$). Ditto's personalized models use Adam (learning rate = $10^{-3}$, batch size = 64, weight decay = $5 \times 10^{-4}$, $\lambda = 0.1$). FedABoost-2 uses AdamW (learning rate = $2 \times 10^{-4}$, batch size = 64, weight decay = $10^{-6}$). FedABoost uses $\eta = 0.01$ and error threshold = 0.5. Total clients: 3,550.
• Figure 3: $F1$ score distributions: Left – MNIST; Right – FEMNIST.