Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Aggregation: Guiding Clients in Heterogeneous Federated Learning

Zijian Wang, Xiaofei Zhang, Xin Zhang, Yukun Liu, Qiong Zhang

TL;DR

The paper tackles statistical heterogeneity in federated learning by turning the server into an intelligent router that routes new queries to the most suitable client. It introduces FedDRM, a unified framework built on a semiparametric density-ratio model and empirical likelihood, jointly learning local predictive models and a client-routing policy via a two-task, EL-based objective. A simple reweighting scheme addresses gradient drift between client-identification and target-class heads, and the authors provide convergence insights under this regime. Empirical results on CIFAR-10/20/100 under non-IID partitions and on a real medical RETINA dataset show that FedDRM improves both system accuracy and routing precision, demonstrating that statistical heterogeneity can be leveraged to build more adaptive, resource-efficient FL systems.

Abstract

Federated learning (FL) is increasingly adopted in domains like healthcare, where data privacy is paramount. A fundamental challenge in these systems is statistical heterogeneity-the fact that data distributions vary significantly across clients (e.g., different hospitals may treat distinct patient demographics). While current FL algorithms focus on aggregating model updates from these heterogeneous clients, the potential of the central server remains under-explored. This paper is motivated by a healthcare scenario: could a central server not only coordinate model training but also guide a new patient to the hospital best equipped for their specific condition? We generalize this idea to propose a novel paradigm for FL systems where the server actively guides the allocation of new tasks or queries to the most appropriate client. To enable this, we introduce a density ratio model and empirical likelihood-based framework that simultaneously addresses two goals: (1) learning effective local models on each client, and (2) finding the best matching client for a new query. Empirical results demonstrate the framework's effectiveness on benchmark datasets, showing improvements in both model accuracy and the precision of client guidance compared to standard FL approaches. This work opens a new direction for building more intelligent and resource-efficient FL systems that leverage heterogeneity as a feature, not just a bug. Code is available at https://github.com/zijianwang0510/FedDRM.git.

Beyond Aggregation: Guiding Clients in Heterogeneous Federated Learning

TL;DR

The paper tackles statistical heterogeneity in federated learning by turning the server into an intelligent router that routes new queries to the most suitable client. It introduces FedDRM, a unified framework built on a semiparametric density-ratio model and empirical likelihood, jointly learning local predictive models and a client-routing policy via a two-task, EL-based objective. A simple reweighting scheme addresses gradient drift between client-identification and target-class heads, and the authors provide convergence insights under this regime. Empirical results on CIFAR-10/20/100 under non-IID partitions and on a real medical RETINA dataset show that FedDRM improves both system accuracy and routing precision, demonstrating that statistical heterogeneity can be leveraged to build more adaptive, resource-efficient FL systems.

Abstract

Federated learning (FL) is increasingly adopted in domains like healthcare, where data privacy is paramount. A fundamental challenge in these systems is statistical heterogeneity-the fact that data distributions vary significantly across clients (e.g., different hospitals may treat distinct patient demographics). While current FL algorithms focus on aggregating model updates from these heterogeneous clients, the potential of the central server remains under-explored. This paper is motivated by a healthcare scenario: could a central server not only coordinate model training but also guide a new patient to the hospital best equipped for their specific condition? We generalize this idea to propose a novel paradigm for FL systems where the server actively guides the allocation of new tasks or queries to the most appropriate client. To enable this, we introduce a density ratio model and empirical likelihood-based framework that simultaneously addresses two goals: (1) learning effective local models on each client, and (2) finding the best matching client for a new query. Empirical results demonstrate the framework's effectiveness on benchmark datasets, showing improvements in both model accuracy and the precision of client guidance compared to standard FL approaches. This work opens a new direction for building more intelligent and resource-efficient FL systems that leverage heterogeneity as a feature, not just a bug. Code is available at https://github.com/zijianwang0510/FedDRM.git.

Paper Structure

This paper contains 29 sections, 4 theorems, 55 equations, 14 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. FedDRM: Guiding Clients in Heterogeneous FL
  3. Probabilistic description of statistical heterogeneity
  4. A semiparametric density ratio model
  5. A surprisingly simple dual loss
  6. Optimization algorithm
  7. Experiments on Benchmark Datasets
  8. Experiment Settings
  9. Evaluation Protocol
  10. Main Results
  11. Sensitivity Analysis
  12. Experiment on Real Medical Dataset
  13. Conclusion
  14. The Use of Large Language Models (LLMs)
  15. Density ratio model examples
  16. ...and 14 more sections

Key Result

Theorem 2.1

With eq:same_client_drm and eq:covariate_shift, the marginal distributions of $X$ also satisfy the DRM: where $\gamma_i^{\dagger} = \gamma_i + \log(\pi_{i1}/\pi_{01})$ for all $i\in[m]$, and $P_{X}^{(0)}$ an unspecified reference measure.

Figures (14)

  • Figure 1: FL server as an intelligent router: Leveraging learned data distributions to direct queries to the most specialized client, rather than applying a global model for diagnosis.
  • Figure 2: Network architecture. Gray blocks are shared among all clients, while colored blocks are specific to each client.
  • Figure 3: Relative gradient drift.
  • Figure 4: Client & image accuracy trade-off on CIFAR-10 under the Dir-0.3 setting.
  • Figure 5: Influence of covariate shift intensity on CIFAR-10 under the Dir-0.3 setting.
  • ...and 9 more figures

Theorems & Definitions (10)

  • Theorem 2.1
  • Theorem 2.2: Dual form
  • Remark 2.3: Beyond covariate shift
  • Remark 2.4: Guiding new queries
  • Theorem 2.5
  • Example B.1: Normal distribution
  • Example B.2: Gamma distribution
  • proof
  • Lemma E.1: Asymptotic normality
  • proof