Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decentralized Proactive Model Offloading and Resource Allocation for Split and Federated Learning

Binbin Huang, Hailiang Zhao, Lingbin Wang, Wenzhuo Qian, Yuyu Yin, Shuiguang Deng

TL;DR

A decentralized proactive model offloading and resource allocation (DP-MORA) scheme is proposed, empowering each end device to determine its cut layer and resource requirements based on its local multidimensional training configuration, without knowledge of other devices’ configurations.

Abstract

In the resource-constrained IoT-edge computing environment, Split Federated (SplitFed) learning is implemented to enhance training efficiency. This method involves each terminal device dividing its full DNN model at a designated layer into a device-side model and a server-side model, then offloading the latter to the edge server. However, existing research overlooks four critical issues as follows: (1) the heterogeneity of end devices' resource capacities and the sizes of their local data samples impact training efficiency; (2) the influence of the edge server's computation and network resource allocation on training efficiency; (3) the data leakage risk associated with the offloaded server-side sub-model; (4) the privacy drawbacks of current centralized algorithms. Consequently, proactively identifying the optimal cut layer and server resource requirements for each end device to minimize training latency while adhering to data leakage risk rate constraint remains a challenging issue. To address these problems, this paper first formulates the latency and data leakage risk of training DNN models using Split Federated learning. Next, we frame the Split Federated learning problem as a mixed-integer nonlinear programming challenge. To tackle this, we propose a decentralized Proactive Model Offloading and Resource Allocation (DP-MORA) scheme, empowering each end device to determine its cut layer and resource requirements based on its local multidimensional training configuration, without knowledge of other devices' configurations. Extensive experiments on two real-world datasets demonstrate that the DP-MORA scheme effectively reduces DNN model training latency, enhances training efficiency, and complies with data leakage risk constraints compared to several baseline algorithms across various experimental settings.

Decentralized Proactive Model Offloading and Resource Allocation for Split and Federated Learning

TL;DR

A decentralized proactive model offloading and resource allocation (DP-MORA) scheme is proposed, empowering each end device to determine its cut layer and resource requirements based on its local multidimensional training configuration, without knowledge of other devices’ configurations.

Abstract

In the resource-constrained IoT-edge computing environment, Split Federated (SplitFed) learning is implemented to enhance training efficiency. This method involves each terminal device dividing its full DNN model at a designated layer into a device-side model and a server-side model, then offloading the latter to the edge server. However, existing research overlooks four critical issues as follows: (1) the heterogeneity of end devices' resource capacities and the sizes of their local data samples impact training efficiency; (2) the influence of the edge server's computation and network resource allocation on training efficiency; (3) the data leakage risk associated with the offloaded server-side sub-model; (4) the privacy drawbacks of current centralized algorithms. Consequently, proactively identifying the optimal cut layer and server resource requirements for each end device to minimize training latency while adhering to data leakage risk rate constraint remains a challenging issue. To address these problems, this paper first formulates the latency and data leakage risk of training DNN models using Split Federated learning. Next, we frame the Split Federated learning problem as a mixed-integer nonlinear programming challenge. To tackle this, we propose a decentralized Proactive Model Offloading and Resource Allocation (DP-MORA) scheme, empowering each end device to determine its cut layer and resource requirements based on its local multidimensional training configuration, without knowledge of other devices' configurations. Extensive experiments on two real-world datasets demonstrate that the DP-MORA scheme effectively reduces DNN model training latency, enhances training efficiency, and complies with data leakage risk constraints compared to several baseline algorithms across various experimental settings.
Paper Structure (18 sections, 38 equations, 8 figures, 4 tables, 2 algorithms)

This paper contains 18 sections, 38 equations, 8 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Efficiency-oriented split federated learning
  4. Efficiency and privacy-oriented split federated learning
  5. System model
  6. System architecture
  7. The latency of DNN model training adopting split federated learning
  8. The data leakage risk of DNN model training adopting split federated learning
  9. Regression-based modeling methodology
  10. Problem formulation
  11. Proposed DP-MORA optimization algorithm
  12. Theoretical analysis
  13. The convergence and optimality of subproblems
  14. The convergence of global problem
  15. Performance evaluation
  16. ...and 3 more sections

Figures (8)

  • Figure 1: System architecture for split federated learning
  • Figure 2: Per-round training latencies of different approaches over ResNet18 (left) and ResNet34 (right) when data leakage risk rate constraint is 0.5
  • Figure 3: Accuracies of different approaches over ResNet 18 on cifar10 dataset
  • Figure 4: Accuracies of different approaches over ResNet 34 on cifar10 dataset
  • Figure 5: Per-round training latency of different approaches over ResNet 18 (left) and ResNet 34 (right) with different data leakage risk rate constraints
  • ...and 3 more figures