Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Client Driven Federated Learning

Songze Li, Chenqing Zhu

TL;DR

This paper tackles the challenge of asynchronous distribution shifts in federated learning by introducing Client-Driven Federated Learning (CDFL), where clients autonomously decide when to update and the server maintains multiple cluster models to tailor updates. The server performs distribution estimation using client uploads, updates cluster models with controlled updates, and sends a personalized model back to the client, all while preserving lightweight client computation and communication. The authors provide a convergence analysis under standard assumptions and validate the approach with extensive experiments on rotated image datasets and digit recognition tasks, showing improved client and cluster performance, reduced overhead, and superior distribution estimation than baselines. Overall, CDFL offers a practical, autonomous, and efficient framework for rapid adaptation to evolving data distributions in distributed settings.

Abstract

Conventional federated learning (FL) frameworks follow a server-driven model where the server determines session initiation and client participation, which faces challenges in accommodating clients' asynchronous needs for model updates. We introduce Client-Driven Federated Learning (CDFL), a novel FL framework that puts clients at the driving role. In CDFL, each client independently and asynchronously updates its model by uploading the locally trained model to the server and receiving a customized model tailored to its local task. The server maintains a repository of cluster models, iteratively refining them using received client models. Our framework accommodates complex dynamics in clients' data distributions, characterized by time-varying mixtures of cluster distributions, enabling rapid adaptation to new tasks with superior performance. In contrast to traditional clustered FL protocols that send multiple cluster models to a client to perform distribution estimation, we propose a paradigm that offloads the estimation task to the server and only sends a single model to a client, and novel strategies to improve estimation accuracy. We provide a theoretical analysis of CDFL's convergence. Extensive experiments across various datasets and system settings highlight CDFL's substantial advantages in model performance and computation efficiency over baselines.

Towards Client Driven Federated Learning

TL;DR

This paper tackles the challenge of asynchronous distribution shifts in federated learning by introducing Client-Driven Federated Learning (CDFL), where clients autonomously decide when to update and the server maintains multiple cluster models to tailor updates. The server performs distribution estimation using client uploads, updates cluster models with controlled updates, and sends a personalized model back to the client, all while preserving lightweight client computation and communication. The authors provide a convergence analysis under standard assumptions and validate the approach with extensive experiments on rotated image datasets and digit recognition tasks, showing improved client and cluster performance, reduced overhead, and superior distribution estimation than baselines. Overall, CDFL offers a practical, autonomous, and efficient framework for rapid adaptation to evolving data distributions in distributed settings.

Abstract

Conventional federated learning (FL) frameworks follow a server-driven model where the server determines session initiation and client participation, which faces challenges in accommodating clients' asynchronous needs for model updates. We introduce Client-Driven Federated Learning (CDFL), a novel FL framework that puts clients at the driving role. In CDFL, each client independently and asynchronously updates its model by uploading the locally trained model to the server and receiving a customized model tailored to its local task. The server maintains a repository of cluster models, iteratively refining them using received client models. Our framework accommodates complex dynamics in clients' data distributions, characterized by time-varying mixtures of cluster distributions, enabling rapid adaptation to new tasks with superior performance. In contrast to traditional clustered FL protocols that send multiple cluster models to a client to perform distribution estimation, we propose a paradigm that offloads the estimation task to the server and only sends a single model to a client, and novel strategies to improve estimation accuracy. We provide a theoretical analysis of CDFL's convergence. Extensive experiments across various datasets and system settings highlight CDFL's substantial advantages in model performance and computation efficiency over baselines.
Paper Structure (16 sections, 1 theorem, 26 equations, 22 figures, 2 tables, 2 algorithms)

This paper contains 16 sections, 1 theorem, 26 equations, 22 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Definition
  4. Client-driven Federated Learning
  5. Client Update
  6. Server Update
  7. Convergence Analysis
  8. Experiments
  9. Setup
  10. Results
  11. Ablation Study
  12. Conclusion
  13. Data Structures, Model Structures and Hyper-parameters
  14. Proof of Theorem \ref{['theorem 1']}
  15. Additional Experiment Results
  16. ...and 1 more sections

Key Result

Theorem 1

For a client with model $v$ and a cluster $k$, we consider consecutive $S_k$ epochs, such that in each epoch the data of the client contains an non-zero component of $P_k$, and the client and cluster $k$ updates $v$ and $w_k$ respectively as in Algorithm alg:CDFL, then with the above assumptions, ch where $w_k^0$ and $w_k^{S_k}$ denotes cluster $k$'s initial model and the model after $S_k$ updates

Figures (22)

  • Figure 1: High-level view of CDFL.
  • Figure 2: CDFL workflow. Client $m$ uploads model and epoch index of last model update $(v_m, \tau)$ to the server. Server performs distribution estimation, derives cluster updating parameters to update the cluster models, and finally constructs an aggregated model $u_m^t$ to send back to the client. Note here as the client's distribution is estimated not to contain $P_2$, cluster 2's model is not updated and not used in computing $u_m^t$.
  • Figure 3: Average accuracy of clients and clusters over time. For client accuracy, in each session all the clients have updated their models for at least once; for cluster accuracy, exact one client updates its model with the sever in each epoch.
  • Figure 4: Average accuracy of clients and clusters on Digit Recognition task over time.
  • Figure 5: (a) Estimated distributions in the MiniImagenet (6 clusters) experiment; (b) KL-divergence between the true distribution and the distribution estimated by CDFL and FedSoft-Async. FM($k$) denotes FashionMNIST ($k$ clusters), Ci as CIFAR-100, M-I as MiniImagenet-100.
  • ...and 17 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof