Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Primal-Dual Algorithm for Hybrid Federated Learning

Tom Overman, Garrett Blum, Diego Klabjan

TL;DR

This paper addresses hybrid federated learning, where clients possess only subsets of samples and features, by introducing HyFDCA, a provably convergent primal-dual algorithm based on Fenchel duality. The method enables local dual coordinate updates, secure inner-product computations, and server-side aggregation to recover a global model that matches centralized-training outcomes under realistic participation patterns. The authors provide convergence proofs across complete, horizontal with random participation, and vertical/incomplete participation regimes, and demonstrate empirical gains over FedAvg and HyFEM on multiple datasets while outlining privacy protections via encryption. The work offers a practical, theoretically grounded framework for doubly distributed data in FL and lays groundwork for privacy-preserving, scalable distributed optimization in hybrid settings.

Abstract

Very few methods for hybrid federated learning, where clients only hold subsets of both features and samples, exist. Yet, this scenario is extremely important in practical settings. We provide a fast, robust algorithm for hybrid federated learning that hinges on Fenchel Duality. We prove the convergence of the algorithm to the same solution as if the model is trained centrally in a variety of practical regimes. Furthermore, we provide experimental results that demonstrate the performance improvements of the algorithm over a commonly used method in federated learning, FedAvg, and an existing hybrid FL algorithm, HyFEM. We also provide privacy considerations and necessary steps to protect client data.

A Primal-Dual Algorithm for Hybrid Federated Learning

TL;DR

This paper addresses hybrid federated learning, where clients possess only subsets of samples and features, by introducing HyFDCA, a provably convergent primal-dual algorithm based on Fenchel duality. The method enables local dual coordinate updates, secure inner-product computations, and server-side aggregation to recover a global model that matches centralized-training outcomes under realistic participation patterns. The authors provide convergence proofs across complete, horizontal with random participation, and vertical/incomplete participation regimes, and demonstrate empirical gains over FedAvg and HyFEM on multiple datasets while outlining privacy protections via encryption. The work offers a practical, theoretically grounded framework for doubly distributed data in FL and lays groundwork for privacy-preserving, scalable distributed optimization in hybrid settings.

Abstract

Very few methods for hybrid federated learning, where clients only hold subsets of both features and samples, exist. Yet, this scenario is extremely important in practical settings. We provide a fast, robust algorithm for hybrid federated learning that hinges on Fenchel Duality. We prove the convergence of the algorithm to the same solution as if the model is trained centrally in a variety of practical regimes. Furthermore, we provide experimental results that demonstrate the performance improvements of the algorithm over a commonly used method in federated learning, FedAvg, and an existing hybrid FL algorithm, HyFEM. We also provide privacy considerations and necessary steps to protect client data.
Paper Structure (22 sections, 5 theorems, 75 equations, 8 figures, 8 tables, 5 algorithms)

This paper contains 22 sections, 5 theorems, 75 equations, 8 figures, 8 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Primal-Dual Algorithm
  4. The HyFDCA Algorithm
  5. Convergence Analysis
  6. Hybrid Federated Setting with Complete Client Participation
  7. Horizontal Federated Setting with Random Subsets of Available Clients
  8. Vertical Federated Setting with Incomplete Client Participation
  9. Experimental Results
  10. Implementation
  11. Results
  12. Convergence Proofs
  13. Proof of Theorem 1
  14. Proof of Theorem 2
  15. Proof of Theorem 3
  16. ...and 7 more sections

Key Result

Theorem 4.4

If Assumptions assum1-assum3 are met, $\gamma_t=1$, and $c_k=N_k/N$ where $N_k$ is the number of samples on client $k$, then Algorithm mainalgo results in the bound on the dual suboptimality gap, $\mathbb{E}[\varepsilon_D^t] \leq (1-\frac{s_t H}{N}) \mathbb{E}[\varepsilon_D^{t-1}] +\frac{s_t^2 H}{N}

Figures (8)

  • Figure 1: Flowchart of HyFDCA. Each vertical arrow represents a communication of some information between clients and the server.
  • Figure 2: Diagram of a possible data partitioning that follows Assumption \ref{['assum3']} for the convergence proof. Each client is shaded in a different color.
  • Figure 3: Comparison of HyFDCA, FedAvg, & HyFEM over constant number of outer iterations across client-fraction settings.
  • Figure 4: Average time costs of components of each outer iteration for HyFDCA, FedAvg, and HyFEM.
  • Figure 5: Comparison of HyFDCA, FedAvg, and HyFEM over constant parallel-equivalent run-time in varying client-fraction settings.
  • ...and 3 more figures

Theorems & Definitions (10)

  • Theorem 4.4
  • Theorem 4.5
  • Theorem 4.8
  • Theorem 4.11
  • Lemma 6.1
  • proof
  • proof
  • proof
  • proof
  • proof