Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning

Chuiyang Meng, Ming Tang, Vincent W. S. Wong

TL;DR

This paper proposes an adaptive split federated learning (ASFL) framework over wireless networks, and proposes an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to solve the problem iteratively.

Abstract

Federated learning (FL) enables multiple clients to collaboratively train a machine learning model without sharing their raw data. However, the limited computation resources of the clients may result in a high delay and energy consumption on training. In this paper, we propose an adaptive split federated learning (ASFL) framework over wireless networks. ASFL exploits the computation resources of the central server to train part of the model and enables adaptive model splitting as well as resource allocation during training. To optimize the learning performance (i.e., convergence rate) and efficiency (i.e., delay and energy consumption) of ASFL, we theoretically analyze the convergence rate and formulate a joint learning performance and resource allocation optimization problem. Solving this problem is challenging due to the long-term delay and energy consumption constraints as well as the coupling of the model splitting and resource allocation decisions. We propose an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to solve the problem iteratively. Experimental results show that when compared with five baseline schemes, our proposed ASFL framework converges faster and reduces the total delay and energy consumption by up to 75% and 80%, respectively.

ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning

TL;DR

This paper proposes an adaptive split federated learning (ASFL) framework over wireless networks, and proposes an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to solve the problem iteratively.

Abstract

Federated learning (FL) enables multiple clients to collaboratively train a machine learning model without sharing their raw data. However, the limited computation resources of the clients may result in a high delay and energy consumption on training. In this paper, we propose an adaptive split federated learning (ASFL) framework over wireless networks. ASFL exploits the computation resources of the central server to train part of the model and enables adaptive model splitting as well as resource allocation during training. To optimize the learning performance (i.e., convergence rate) and efficiency (i.e., delay and energy consumption) of ASFL, we theoretically analyze the convergence rate and formulate a joint learning performance and resource allocation optimization problem. Solving this problem is challenging due to the long-term delay and energy consumption constraints as well as the coupling of the model splitting and resource allocation decisions. We propose an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to solve the problem iteratively. Experimental results show that when compared with five baseline schemes, our proposed ASFL framework converges faster and reduces the total delay and energy consumption by up to 75% and 80%, respectively.
Paper Structure (29 sections, 58 equations, 12 figures, 3 tables, 2 algorithms)

This paper contains 29 sections, 58 equations, 12 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. System Model
  3. Learning Model
  4. Delay and Energy Consumption Model
  5. Delay
  6. Energy Consumption
  7. Theoretical Analysis and Problem Formulation
  8. Theoretical Analysis
  9. Problem Formulation
  10. OOE-BCD Algorithm
  11. Model Splitting Subproblem
  12. Online Optimization Algorithm
  13. Stability Analysis
  14. RB Allocation Subproblem
  15. Transmit Power Allocation Subproblem
  16. ...and 14 more sections

Figures (12)

  • Figure 1: Illustration of our proposed ASFL framework.
  • Figure 2: An example of the adaptive model splitting process in stage 1 for a model with 7 layers. A client sends its fourth layer to the central server at the beginning of the $r$-th training round.
  • Figure 3: An illustration of the delay in one ASFL training round.
  • Figure 4: Comparison of the average testing accuracy versus (a) total delay and (b) total energy consumption on training.
  • Figure 5: Comparison of the average packet error rate and average long-term model discrepancies.
  • ...and 7 more figures

Theorems & Definitions (7)

  • proof
  • proof
  • proof
  • proof
  • proof
  • proof
  • proof