Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

I-SplitEE: Image classification in Split Computing DNNs with Early Exits

Divya Jyoti Bajpai, Aastha Jaiswal, Manjesh Kumar Hanawal

TL;DR

This work tackles the challenge of deploying large DNNs on edge devices by unifying split computing with early exits. It introduces I-SplitEE, an online unsupervised Multi-Armed Bandit-based method that dynamically selects the optimal splitting layer and decides whether to infer on the edge or offload to the cloud, adapting to distributional shifts caused by distortions. Empirical results on Caltech-256 and CIFAR-10 with MobileNetV2 show substantial cost reductions (over $55\%$) with only modest accuracy degradation (up to $5\%$), demonstrating robustness to distortion and varying offloading costs. The approach enables practical, distortion-aware edge inference with scalable edge-cloud resource utilization and minimal performance trade-offs.

Abstract

The recent advances in Deep Neural Networks (DNNs) stem from their exceptional performance across various domains. However, their inherent large size hinders deploying these networks on resource-constrained devices like edge, mobile, and IoT platforms. Strategies have emerged, from partial cloud computation offloading (split computing) to integrating early exits within DNN layers. Our work presents an innovative unified approach merging early exits and split computing. We determine the 'splitting layer', the optimal depth in the DNN for edge device computations, and whether to infer on edge device or be offloaded to the cloud for inference considering accuracy, computational efficiency, and communication costs. Also, Image classification faces diverse environmental distortions, influenced by factors like time of day, lighting, and weather. To adapt to these distortions, we introduce I-SplitEE, an online unsupervised algorithm ideal for scenarios lacking ground truths and with sequential data. Experimental validation using Caltech-256 and Cifar-10 datasets subjected to varied distortions showcases I-SplitEE's ability to reduce costs by a minimum of 55% with marginal performance degradation of at most 5%.

I-SplitEE: Image classification in Split Computing DNNs with Early Exits

TL;DR

This work tackles the challenge of deploying large DNNs on edge devices by unifying split computing with early exits. It introduces I-SplitEE, an online unsupervised Multi-Armed Bandit-based method that dynamically selects the optimal splitting layer and decides whether to infer on the edge or offload to the cloud, adapting to distributional shifts caused by distortions. Empirical results on Caltech-256 and CIFAR-10 with MobileNetV2 show substantial cost reductions (over ) with only modest accuracy degradation (up to ), demonstrating robustness to distortion and varying offloading costs. The approach enables practical, distortion-aware edge inference with scalable edge-cloud resource utilization and minimal performance trade-offs.

Abstract

The recent advances in Deep Neural Networks (DNNs) stem from their exceptional performance across various domains. However, their inherent large size hinders deploying these networks on resource-constrained devices like edge, mobile, and IoT platforms. Strategies have emerged, from partial cloud computation offloading (split computing) to integrating early exits within DNN layers. Our work presents an innovative unified approach merging early exits and split computing. We determine the 'splitting layer', the optimal depth in the DNN for edge device computations, and whether to infer on edge device or be offloaded to the cloud for inference considering accuracy, computational efficiency, and communication costs. Also, Image classification faces diverse environmental distortions, influenced by factors like time of day, lighting, and weather. To adapt to these distortions, we introduce I-SplitEE, an online unsupervised algorithm ideal for scenarios lacking ground truths and with sequential data. Experimental validation using Caltech-256 and Cifar-10 datasets subjected to varied distortions showcases I-SplitEE's ability to reduce costs by a minimum of 55% with marginal performance degradation of at most 5%.
Paper Structure (10 sections, 3 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 10 sections, 3 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Problem Setup
  4. Algorithm
  5. Experiments
  6. Baselines
  7. Impact of change in distortion
  8. Analysis over different overheads
  9. Results
  10. Conclusion

Figures (4)

  • Figure 1: Split computing combined with early exits, where the DNN is split into two parts with an additional advantage of inference on mobile device.
  • Figure 2: Effect of different offloading costs over the total cost. The cost is mostly linear but with some deviations.
  • Figure 3: Effect of different offloading costs over the accuracy. The accuracy remains constant, however, the changes might be due to randomness in the dataset.
  • Figure 4: Effect of added noise on confidence. These confidence values are the final layer.