FedGCS: A Generative Framework for Efficient Client Selection in Federated Learning via Gradient-based Optimization

TL;DR

FedGCS tackles the enduring challenges of federated learning—statistical and system heterogeneity and high energy costs—by reframing client selection as a generative decision problem. It builds a continuous representation space through an encoder-evaluator-decoder trio trained on selection-score data collected from classical heuristic and RL baselines, then performs gradient-based optimization in this space and generates the final subset with beam search. The approach yields substantial improvements in model accuracy, especially under non-IID conditions, and demonstrates faster convergence with lower energy consumption compared to traditional methods. These results suggest a scalable, data-driven, and generalizable paradigm for efficient client selection in FL with potential to influence practical deployment in diverse, heterogeneous environments.

Abstract

Federated Learning faces significant challenges in statistical and system heterogeneity, along with high energy consumption, necessitating efficient client selection strategies. Traditional approaches, including heuristic and learning-based methods, fall short of addressing these complexities holistically. In response, we propose FedGCS, a novel generative client selection framework that innovatively recasts the client selection process as a generative task. Drawing inspiration from the methodologies used in large language models, FedGCS efficiently encodes abundant decision-making knowledge within a continuous representation space, enabling efficient gradient-based optimization to search for optimal client selection that will be finally output via generation. The framework comprises four steps: (1) automatic collection of diverse "selection-score" pair data using classical client selection methods; (2) training an encoder-evaluator-decoder framework on this data to construct a continuous representation space; (3) employing gradient-based optimization in this space for optimal client selection; (4) generating the final optimal client selection via using beam search for the well-trained decoder. FedGCS outperforms traditional methods by being more comprehensive, generalizable, and efficient, simultaneously optimizing for model performance, latency, and energy consumption. The effectiveness of FedGCS is proven through extensive experimental analyses.

Figures (9)

• Figure 1: (a) Right: heuristic-based selection, which utilizes manually developed heuristic policies to select participating devices. (a) Left: learning-based selection, which employs RL to train continuously self-optimizing agents for optimal client selection. (b): FedGCS formulates client selection as a generative task.
• Figure 2: Left: reward convergence curve of Value Decomposition Network VDN for RL agent in two client selection tasks. Right: ternary depiction of the 3D optimization space.
• Figure 3: Framework overview of FedGCS: (1) efficiently collecting sufficient, diverse, comprehensive and high-quality training data; (2) preserving the knowledge of classical client selection methods into a global continuous representation space; (3) searching for better representation in the learned space via gradient-based optimization; (4) outputting the optimal device subset via generation.
• Figure 4: Efficiency comparison of FedGCS with three SOTA baselines to train ResNet18 on CIFAR10 (IID). Left: ToA. Right: EoA.
• Figure 5: Generalization ability of FedGCS.