Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning the Optimal Path and DNN Partition for Collaborative Edge Inference

Yin Huang, Letian Zhang, Jie Xu

TL;DR

This paper proposes B-EXPUCB, a new algorithm that combines ideas from blocked EXP3 and LinUCB, and shows that it achieves sublinear regret, and outperforms existing methods in collaborative edge inference.

Abstract

Recent advancements in Deep Neural Networks (DNNs) have catalyzed the development of numerous intelligent mobile applications and services. However, they also introduce significant computational challenges for resource-constrained mobile devices. To address this, collaborative edge inference has been proposed. This method involves partitioning a DNN inference task into several subtasks and distributing these across multiple network nodes. Despite its potential, most current approaches presume known network parameters -- like node processing speeds and link transmission rates -- or rely on a fixed sequence of nodes for processing the DNN subtasks. In this paper, we tackle a more complex scenario where network parameters are unknown and must be learned, and multiple network paths are available for distributing inference tasks. Specifically, we explore the learning problem of selecting the optimal network path and assigning DNN layers to nodes along this path, considering potential security threats and the costs of switching paths. We begin by deriving structural insights from the DNN layer assignment with complete network information, which narrows down the decision space and provides crucial understanding of optimal assignments. We then cast the learning problem with incomplete network information as a novel adversarial group linear bandits problem with switching costs, featuring rewards generation through a combined stochastic and adversarial process. We introduce a new bandit algorithm, B-EXPUCB, which combines elements of the classical blocked EXP3 and LinUCB algorithms, and demonstrate its sublinear regret. Extensive simulations confirm B-EXPUCB's superior performance in learning for collaborative edge inference over existing algorithms.

Learning the Optimal Path and DNN Partition for Collaborative Edge Inference

TL;DR

This paper proposes B-EXPUCB, a new algorithm that combines ideas from blocked EXP3 and LinUCB, and shows that it achieves sublinear regret, and outperforms existing methods in collaborative edge inference.

Abstract

Recent advancements in Deep Neural Networks (DNNs) have catalyzed the development of numerous intelligent mobile applications and services. However, they also introduce significant computational challenges for resource-constrained mobile devices. To address this, collaborative edge inference has been proposed. This method involves partitioning a DNN inference task into several subtasks and distributing these across multiple network nodes. Despite its potential, most current approaches presume known network parameters -- like node processing speeds and link transmission rates -- or rely on a fixed sequence of nodes for processing the DNN subtasks. In this paper, we tackle a more complex scenario where network parameters are unknown and must be learned, and multiple network paths are available for distributing inference tasks. Specifically, we explore the learning problem of selecting the optimal network path and assigning DNN layers to nodes along this path, considering potential security threats and the costs of switching paths. We begin by deriving structural insights from the DNN layer assignment with complete network information, which narrows down the decision space and provides crucial understanding of optimal assignments. We then cast the learning problem with incomplete network information as a novel adversarial group linear bandits problem with switching costs, featuring rewards generation through a combined stochastic and adversarial process. We introduce a new bandit algorithm, B-EXPUCB, which combines elements of the classical blocked EXP3 and LinUCB algorithms, and demonstrate its sublinear regret. Extensive simulations confirm B-EXPUCB's superior performance in learning for collaborative edge inference over existing algorithms.
Paper Structure (15 sections, 3 theorems, 23 equations, 10 figures, 1 table, 1 algorithm)

This paper contains 15 sections, 3 theorems, 23 equations, 10 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Algorithm
  5. EXPUCB
  6. Regret Analysis
  7. Application: Collaborative Edge Inference
  8. Background and Problem Formulation
  9. Simulation Setup
  10. Baseline Algorithms
  11. Performance Comparison
  12. Behaviors of EXPUCB
  13. Impact of Time-Varying Task Distributions
  14. Impact of Attacker Strategies
  15. Conclusion

Key Result

Lemma 1

The estimated reward $x^\top \hat{\theta}^t_g$ satisfies the following error bound for all arm $x$, all group $g$ and round $t$ with probability at least $1-\delta$,

Figures (10)

  • Figure 1: Illustration of collaborative edge inference.
  • Figure 2: An example DNN partition and the associated feature vector.
  • Figure 3: Computational workload and intermediate data size of YoLo/ResNet50.
  • Figure 4: Regret achieved by EXPUCB and baselines.
  • Figure 5: Total reward achieved by EXPUCB and baselines.
  • ...and 5 more figures

Theorems & Definitions (6)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Theorem 1
  • proof