Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Emulating Full Participation: An Effective and Fair Client Selection Strategy for Federated Learning

Qingming Li, Juzheng Miao, Puning Zhao, Li Zhou, H. Vicky Zhao, Shouling Ji, Bowen Zhou, Furui Liu

TL;DR

This work forms the client selection problem as a long-term optimization task, leveraging the Lyapunov function and the submodular nature of the problem to solve it effectively, and proposes two guiding principles that directly tackle the inherent conflict between the two metrics while reinforcing each other.

Abstract

In federated learning, client selection is a critical problem that significantly impacts both model performance and fairness. Prior studies typically treat these two objectives separately, or balance them using simple weighting schemes. However, we observe that commonly used metrics for model performance and fairness often conflict with each other, and a straightforward weighted combination is insufficient to capture their complex interactions. To address this, we first propose two guiding principles that directly tackle the inherent conflict between the two metrics while reinforcing each other. Based on these principles, we formulate the client selection problem as a long-term optimization task, leveraging the Lyapunov function and the submodular nature of the problem to solve it effectively. Experiments show that the proposed method improves both model performance and fairness, guiding the system to converge comparably to full client participation. This improvement can be attributed to the fact that both model performance and fairness benefit from the diversity of the selected clients' data distributions. Our approach adaptively enhances this diversity by selecting clients based on their data distributions, thereby improving both model performance and fairness.

Emulating Full Participation: An Effective and Fair Client Selection Strategy for Federated Learning

TL;DR

This work forms the client selection problem as a long-term optimization task, leveraging the Lyapunov function and the submodular nature of the problem to solve it effectively, and proposes two guiding principles that directly tackle the inherent conflict between the two metrics while reinforcing each other.

Abstract

In federated learning, client selection is a critical problem that significantly impacts both model performance and fairness. Prior studies typically treat these two objectives separately, or balance them using simple weighting schemes. However, we observe that commonly used metrics for model performance and fairness often conflict with each other, and a straightforward weighted combination is insufficient to capture their complex interactions. To address this, we first propose two guiding principles that directly tackle the inherent conflict between the two metrics while reinforcing each other. Based on these principles, we formulate the client selection problem as a long-term optimization task, leveraging the Lyapunov function and the submodular nature of the problem to solve it effectively. Experiments show that the proposed method improves both model performance and fairness, guiding the system to converge comparably to full client participation. This improvement can be attributed to the fact that both model performance and fairness benefit from the diversity of the selected clients' data distributions. Our approach adaptively enhances this diversity by selecting clients based on their data distributions, thereby improving both model performance and fairness.
Paper Structure (33 sections, 5 theorems, 61 equations, 14 figures, 1 table, 2 algorithms)

This paper contains 33 sections, 5 theorems, 61 equations, 14 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Client Selection in Federated Learning
  4. Fairness in Federated Learning
  5. The Proposed Optimization Function
  6. Preliminary of Federated Learning
  7. The Optimization Function
  8. Formulation of Principle I
  9. Formulation of Principle II
  10. The Optimization Function
  11. Transformation Under Lyapunov Optimization
  12. The Proposed LongFed
  13. The Client Selection Strategy
  14. Convergence Analysis
  15. Experiment
  16. ...and 18 more sections

Key Result

Theorem 1

Define and then $\text{DUB}(\mathbb{S}^t)$ serves as an upper bound for $\text{D}(\mathbb{S}^t)$.

Figures (14)

  • Figure 1: Visualization of the client selection results. Clients are represented by blue dots, and selected clients are marked with orange stars. In the non-clustering scenario, clients arranged in a circle are effectively represented by the selected client positioned at the center.
  • Figure 2: Test accuracy under the IID scenario.
  • Figure 3: Test accuracy on FMNIST and CIFAR-10 under three heterogeneous data partitioning settings.
  • Figure 4: Fairness results on the CIFAR-10 dataset in the 2SPC scenario.
  • Figure 5: Fairness results on the FMNIST dataset in the 2SPC scenario.
  • ...and 9 more figures

Theorems & Definitions (11)

  • Theorem 1
  • Definition 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • proof
  • proof
  • proof
  • Lemma 1
  • proof
  • ...and 1 more