Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices

TL;DR

This work addresses the gap in understanding and improving energy efficiency for on-device deep learning on edge devices. It combines a comprehensive energy measurement study across kernels, DNN models, and end-to-end applications on eight devices with the creation of kernel-, model-, and application-level energy datasets, a kernel-level energy predictor trained per kernel type, and two end-user scoring metrics (PCS and IECS) to communicate power and energy performance accessibly. The kernel-level predictor achieves an average accuracy of 86.2% on unseen models, outperforming FLOPs- and BIC-based baselines, underscoring the importance of fine-grained energy data and realistic runtime conditions. Overall, the paper contributes open datasets, a scalable energy prediction approach, and user-friendly scoring to promote transparency and sustainability in edge DL deployment and development, with practical implications for device benchmarking and kernel-level optimization.

Abstract

Today, deep learning optimization is primarily driven by research focused on achieving high inference accuracy and reducing latency. However, the energy efficiency aspect is often overlooked, possibly due to a lack of sustainability mindset in the field and the absence of a holistic energy dataset. In this paper, we conduct a threefold study, including energy measurement, prediction, and efficiency scoring, with an objective to foster transparency in power and energy consumption within deep learning across various edge devices. Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning. This study results in the creation of three extensive energy datasets for edge devices, covering a wide range of kernels, state-of-the-art DNN models, and popular AI applications. Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset. Evaluation results demonstrate the ability of our predictors to provide consistent and accurate energy estimations on unseen DNN models. Lastly, we introduce two scoring metrics, PCS and IECS, developed to convert complex power and energy consumption data of an edge device into an easily understandable manner for edge device end-users. We hope our work can help shift the mindset of both end-users and the research community towards sustainability in edge computing, a principle that drives our research. Find data, code, and more up-to-date information at https://amai-gsu.github.io/DeepEn2023.

Figures (11)

• Figure 1: Comparison of time-granularity between the device's built-in current sensor and external power monitor.
• Figure 2: Comparison between an older snap-type battery connector and a modern FPC connector.
• Figure 3: Measured devices with segregated BMS chips.
• Figure 4: Energy consumption of conv$++$bn$++$relu vs. kernel configurations.
• Figure 5: Comparison of energy consumption of conv$++$bn$++$relu with identical configurations on mobile CPU and GPU ($HW = 1, KS = 1, S = 1$, measured device: Huwei P40 Pro). Using the mobile GPU to execute the kernel does not always save the device's energy compared to using the mobile CPU.