Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sustainable Edge Intelligence Through Energy-Aware Early Exiting

Marcello Bullo, Seifallah Jardak, Pietro Carnelli, Deniz Gündüz

TL;DR

This work proposes energy-adaptive dynamic early exiting (EE) to enable efficient and accurate inference in an EH edge intelligence system and derives an energy-aware EE policy that determines the optimal amount of computational processing on a per-sample basis.

Abstract

Deep learning (DL) models have emerged as a promising solution for the Internet of Things (IoT). However, due to their computational complexity, DL models consume significant amounts of energy, which can rapidly drain the battery and compromise the performance of IoT devices. For sustainable operation, we consider an edge device with a rechargeable battery and energy harvesting (EH) capabilities. In addition to the stochastic nature of the ambient energy source, the harvesting rate is often insufficient to meet the inference energy requirements, leading to drastic performance degradation in energy-agnostic devices. To mitigate this problem, we propose energy-adaptive dynamic early exiting (EE) to enable efficient and accurate inference in an EH edge intelligence system. Our approach derives an energy-aware EE policy that determines the optimal amount of computational processing on a per-sample basis. The proposed policy balances the energy consumption to match the limited incoming energy and achieves continuous availability. Numerical results show that accuracy and service rate are improved up to 25% and 35%, respectively, in comparison with an energy-agnostic policy.

Sustainable Edge Intelligence Through Energy-Aware Early Exiting

TL;DR

This work proposes energy-adaptive dynamic early exiting (EE) to enable efficient and accurate inference in an EH edge intelligence system and derives an energy-aware EE policy that determines the optimal amount of computational processing on a per-sample basis.

Abstract

Deep learning (DL) models have emerged as a promising solution for the Internet of Things (IoT). However, due to their computational complexity, DL models consume significant amounts of energy, which can rapidly drain the battery and compromise the performance of IoT devices. For sustainable operation, we consider an edge device with a rechargeable battery and energy harvesting (EH) capabilities. In addition to the stochastic nature of the ambient energy source, the harvesting rate is often insufficient to meet the inference energy requirements, leading to drastic performance degradation in energy-agnostic devices. To mitigate this problem, we propose energy-adaptive dynamic early exiting (EE) to enable efficient and accurate inference in an EH edge intelligence system. Our approach derives an energy-aware EE policy that determines the optimal amount of computational processing on a per-sample basis. The proposed policy balances the energy consumption to match the limited incoming energy and achieves continuous availability. Numerical results show that accuracy and service rate are improved up to 25% and 35%, respectively, in comparison with an energy-agnostic policy.
Paper Structure (13 sections, 2 theorems, 17 equations, 4 figures)

This paper contains 13 sections, 2 theorems, 17 equations, 4 figures.

Table of Contents

  1. Introduction
  2. System Model
  3. Problem Formulation
  4. System Optimization
  5. Optimal Non-Causal Controller
  6. Feasible Causal Controller
  7. Energy-Agnostic Oracle
  8. Numerical Results
  9. Dataset
  10. DNN architecture
  11. oNCC policies
  12. Controllers comparison
  13. Conclusion

Key Result

Lemma 1

For each $s=(b,h)\in \mathcal{S}, b\geq u(c)$, there exists a threshold $\gamma_s$ on the confidence gain $j\triangleq z^c-z^e$ such that the policy $\pi^*(s)=g^*_s$ defined as dominates any other policy $\pi\in\Pi$, that is $\rho_{\pi^*}\geq\rho_\pi$, $\forall \pi\in\Pi$.

Figures (4)

  • Figure 1: Block diagram of the intelligent edge device with EH capabilities and rechargeable battery.
  • Figure 2: Learned policies over the $\mathcal{D_{\text{est}}}$, when $p_G=0.9$, and (a.) $\lambda_1=0.2$, $\lambda_2=0.7$, and (b.) $p_\beta=0.6$. Solid and dashed lines represent the policy for states $\{ G,\beta \}$, respectively.
  • Figure 3: Effective accuracy $\alpha$, accuracy $\varrho$, and service rate $\tau$ computed over the test set $\mathcal{D}_{\text{test}}$, when $\lambda_1=0.2$, $\lambda_2=0.7$, $p_G= 0.9$, $p_\beta=0.6$, $\varepsilon=1.28$, $T=10^4$, $b_{max}=50$.
  • Figure 4: Mean and confidence intervals of cumulative consumed energy (left) and battery level (right), with the configuration setting of Fig. \ref{['fig:barplot']}. The inset graph (left) shows the behavior of the consumed energy in long-term horizon.

Theorems & Definitions (5)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • proof