Conflict-Aware Client Selection for Multi-Server Federated Learning

TL;DR

Multi-server federated learning often suffers from inter-server contention when overlapping coverage causes multiple servers to select the same clients, leading to bandwidth contention and delayed updates. The paper introduces RL-CRP, a decentralized framework that combines conflict risk prediction via a categorical hidden Markov model with reinforcement learning (Soft Actor-Critic) guided by a fairness-aware reward, plus a water-filling bandwidth allocation step. Key contributions include a CRP mechanism with incremental Baum–Welch updates for sparse histories, a decentralized SAC-based client selection policy that accounts for latency, conflicts, and long-term client participation, and extensive experiments on CIFAR-10 showing reduced conflicts and faster convergence under both IID and non-IID data, with demonstrated scalability to more servers. The approach offers practical gains for scalable, edge-based multi-server FL by mitigating contention while sustaining fair client participation and efficient communication.

Abstract

Federated learning (FL) has emerged as a promising distributed machine learning (ML) that enables collaborative model training across clients without exposing raw data, thereby preserving user privacy and reducing communication costs. Despite these benefits, traditional single-server FL suffers from high communication latency due to the aggregation of models from a large number of clients. While multi-server FL distributes workloads across edge servers, overlapping client coverage and uncoordinated selection often lead to resource contention, causing bandwidth conflicts and training failures. To address these limitations, we propose a decentralized reinforcement learning with conflict risk prediction, named RL CRP, to optimize client selection in multi-server FL systems. Specifically, each server estimates the likelihood of client selection conflicts using a categorical hidden Markov model based on its sparse historical client selection sequence. Then, a fairness-aware reward mechanism is incorporated to promote long-term client participation for minimizing training latency and resource contention. Extensive experiments demonstrate that the proposed RL-CRP framework effectively reduces inter-server conflicts and significantly improves training efficiency in terms of convergence speed and communication cost.

Figures (4)

• Figure 1: Illustration of multi-server FL.
• Figure 2: Test accuracy of baseline algorithms.
• Figure 3: Reward performance of baseline algorithms with different number of servers.
• Figure 4: Conflict performance versus different number of servers.