Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Client-Centric Federated Adaptive Optimization

Jianhui Sun, Xidong Wu, Heng Huang, Aidong Zhang

TL;DR

The paper addresses the gap between idealized server-centric federated learning and real-world deployments that feature massive, heterogeneous clients, unpredictable availability, and asynchronous computation. It introduces Client-Centric Federated Adaptive Optimization (CC-FL), a framework that allows arbitrary client participation, asynchronous server aggregation, and device-dependent local updates, with server-side adaptive optimization to counter client drift. The authors provide a convergence analysis for nonconvex objectives that achieves a best-known rate under asynchronous, heterogeneous conditions and demonstrate substantial empirical gains over FedAvg baselines across vision and language benchmarks. The work highlights the practical impact of combining client-centric control with adaptive global optimization for scalable, robust federated learning.

Abstract

Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private. With an increasing scale of clients and models, FL encounters two key challenges, client drift due to a high degree of statistical/system heterogeneity, and lack of adaptivity. However, most existing FL research is based on unrealistic assumptions that virtually ignore system heterogeneity. In this paper, we propose Client-Centric Federated Adaptive Optimization, which is a class of novel federated adaptive optimization approaches. We enable several features in this framework such as arbitrary client participation, asynchronous server aggregation, and heterogeneous local computing, which are ubiquitous in real-world FL systems but are missed in most existing works. We provide a rigorous convergence analysis of our proposed framework for general nonconvex objectives, which is shown to converge with the best-known rate. Extensive experiments show that our approaches consistently outperform the baseline by a large margin across benchmarks.

Client-Centric Federated Adaptive Optimization

TL;DR

The paper addresses the gap between idealized server-centric federated learning and real-world deployments that feature massive, heterogeneous clients, unpredictable availability, and asynchronous computation. It introduces Client-Centric Federated Adaptive Optimization (CC-FL), a framework that allows arbitrary client participation, asynchronous server aggregation, and device-dependent local updates, with server-side adaptive optimization to counter client drift. The authors provide a convergence analysis for nonconvex objectives that achieves a best-known rate under asynchronous, heterogeneous conditions and demonstrate substantial empirical gains over FedAvg baselines across vision and language benchmarks. The work highlights the practical impact of combining client-centric control with adaptive global optimization for scalable, robust federated learning.

Abstract

Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private. With an increasing scale of clients and models, FL encounters two key challenges, client drift due to a high degree of statistical/system heterogeneity, and lack of adaptivity. However, most existing FL research is based on unrealistic assumptions that virtually ignore system heterogeneity. In this paper, we propose Client-Centric Federated Adaptive Optimization, which is a class of novel federated adaptive optimization approaches. We enable several features in this framework such as arbitrary client participation, asynchronous server aggregation, and heterogeneous local computing, which are ubiquitous in real-world FL systems but are missed in most existing works. We provide a rigorous convergence analysis of our proposed framework for general nonconvex objectives, which is shown to converge with the best-known rate. Extensive experiments show that our approaches consistently outperform the baseline by a large margin across benchmarks.
Paper Structure (24 sections, 2 theorems, 6 equations, 10 figures, 2 tables)

This paper contains 24 sections, 2 theorems, 6 equations, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Federated Averaging (FedAvg)
  4. Federated Adaptive Optimization
  5. Client-Centric Federated Adaptive Optimization
  6. Limitations of Server-Centric FL
  7. Methodology
  8. Workflow of Algorithm \ref{['alg:cc_fed_adaptive_algs']}
  9. Key Algorithmic Designs
  10. Convergence Analysis
  11. Experiments
  12. Experimental Setting
  13. Performances on Benchmark Datasets
  14. Ease of Hyperparameter Tuning
  15. Related Work
  16. ...and 9 more sections

Key Result

Theorem 4.1

Suppose $\{f_i\}_{i=1}^n$ fulfills Assumptions smoothness_assumption-bounded_global_assumption. Suppose the maximum delay is bounded, i.e., $\tau_{t,i}\leq\tau<\infty$ for any $i\in\mathcal{S}_t$ and $t\in\{1,\dots,T\}$. Under the condition , where $K_{t,\text{max}} =\max_{i\in\mathcal{S}_t}K_{t,i}$, and suppose $H_1 \eta_l^2+H_2\eta_l\le\epsilon^2$, where $H_1\triangleq 2 \eta^2 L^2 \tau^2$, $H_

Figures (10)

  • Figure 1: Training and testing curves for various CC-Federated Adaptive Optimizers (ResNet on CIFAR-10) under different Concentration Parameters $\alpha$.
  • Figure 2: Testing curve for various CC-Federated Adaptive Optimizers on different settings of $\tau$, $R$, datasets, and architectures.
  • Figure 3: Hyperparameter sensitivity of CC-Federated Adaptive Optimizers. Note we do not plot CC-FedAdagrad in Figure \ref{['subfig:gamma_sensitivity']}, as there is no $\gamma$ in CC-FedAdagrad.
  • Figure 4: A toy example of a two-client FL setting. There is a mismatch between the ideal direction towards global optimum (green dashed arrow) and the actual search direction (red solid line) if the two clients take a different number of local updates.
  • Figure 5: Training and testing curves for various CC-Federated Adaptive Optimizers (ResNet on CIFAR-10) with $\tau=10$.
  • ...and 5 more figures

Theorems & Definitions (5)

  • Theorem 4.1: Convergence of CC-Federated Adaptive Optimization
  • Corollary 4.1.1: Convergence Rate of Client-Centric Federated Adaptive Optimization
  • Remark 4.1.1: How Random Delay Impacts Convergence?
  • Remark 4.1.2: Convergence Rate of Arbitrary Participation
  • Remark 4.1.3: Linear Speedup w.r.t $m,K$