Entropy-driven Fair and Effective Federated Learning
Lin Wang, Zhichao Wang, Ye Shi, Sai Praneeth Karimireddy, Xiaoying Tang
TL;DR
This work tackles fairness in Federated Learning under heterogeneous client data by balancing client performance without sacrificing global accuracy. It introduces FedEBA+, a bi-level optimization framework combining entropy-based fair aggregation with model and gradient alignment, yielding an analytic inner solution for aggregation weights and adaptive outer-loop updates. Theoretical results establish convergence in nonconvex FL and fairness improvements across generalized linear and strongly convex settings, while empirical results show reduced performance variance and higher global accuracy on diverse datasets; a communication-efficient variant (Prac-FedEBA+) maintains these gains with FedAvg-like costs. The approach also demonstrates robustness to noisy labels and compatibility with differential privacy, though Byzantine-robust extensions remain for future work, underscoring its practical impact for fair, scalable FL deployment.
Abstract
Federated Learning (FL) enables collaborative model training across distributed devices while preserving data privacy. Nonetheless, the heterogeneity of edge devices often leads to inconsistent performance of the globally trained models, resulting in unfair outcomes among users. Existing federated fairness algorithms strive to enhance fairness but often fall short in maintaining the overall performance of the global model, typically measured by the average accuracy across all clients. To address this issue, we propose a novel algorithm that leverages entropy-based aggregation combined with model and gradient alignments to simultaneously optimize fairness and global model performance. Our method employs a bi-level optimization framework, where we derive an analytic solution to the aggregation probability in the inner loop, making the optimization process computationally efficient. Additionally, we introduce an innovative alignment update and an adaptive strategy in the outer loop to further balance global model's performance and fairness. Theoretical analysis indicates that our approach guarantees convergence even in non-convex FL settings and demonstrates significant fairness improvements in generalized regression and strongly convex models. Empirically, our approach surpasses state-of-the-art federated fairness algorithms, ensuring consistent performance among clients while improving the overall performance of the global model.