FedDES: Graph-Based Dynamic Ensemble Selection for Personalized Federated Learning

Abstract

Statistical heterogeneity in Federated Learning (FL) often leads to negative transfer, where a single global model fails to serve diverse client distributions. Personalized federated learning (pFL) aims to address this by tailoring models to individual clients. However, under most existing pFL approaches, clients integrate peer client contributions uniformly, which ignores the reality that not all peers are likely to be equally beneficial. Additionally, the potential for personalization at the instance level remains largely unexplored, even though the reliability of different peer models often varies across individual samples within the same client. We introduce FedDES (Federated Dynamic Ensemble Selection), a decentralized pFL framework that achieves instance-level personalization through dynamic ensemble selection. Central to our approach is a Graph Neural Network (GNN) meta-learner trained on a heterogeneous graph modeling interactions between data samples and candidate classifiers. For each test query, the GNN dynamically selects and weights peer client models, forming an ensemble of the most competent classifiers while effectively suppressing contributions from those that are irrelevant or potentially harmful for performance. Experiments on CIFAR-10 and real-world ICU healthcare data demonstrate that FedDES outperforms state-of-the-art pFL baselines in non-IID settings, offering robust protection against negative transfer.

Figures (2)

• Figure 1: Overview of FedDES. Stage 1: Clients independently train heterogeneous base classifiers on their local data and exchange models via peer-to-peer communication, forming a shared classifier pool. Stage 2: Each client evaluates the full classifier pool on its local data to obtain decision-space representations $\mathbf{P}_k$ and meta-labels $\mathbf{Z}_k$, and then constructs a heterogeneous graph in which sample nodes (circles) are linked by decision-space similarity and classifier nodes (triangles) are connected to samples based on local competence. Stage 3: A heterogeneous GATv2 meta-learner is trained to map sample embeddings, refined through message passing over the graph, to per-classifier competence scores. Inference: A new query $x_q$ is projected into the decision space, inserted into the graph, and processed by the trained GNN. The resulting competence scores $\sigma(\mathbf{s}_i)$ determine which classifiers are selected (scores above 0.5) and their relative voting strength in the final weighted ensemble prediction $\hat{y}_q$.
• Figure 2: Meta-learner selection scores vs. home client class frequency on CIFAR-10 across six heterogeneity settings. Each point represents one (classifier, target class) pair averaged across all clients. Red points indicate local classifiers; gray points indicate non-local classifiers. Dashed line shows linear trend. $\rho$ denotes Spearman correlation. Top row: $\alpha{=}1$; bottom row: $\alpha{=}10$. Columns vary $C \in \{3, 5, 7\}$.