Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Energy-Aware Deep Learning on Resource-Constrained Hardware

Josh Millar, Hamed Haddadi, Anil Madhavapeddy

TL;DR

This survey analyzes energy-aware deep learning for IoT and edge devices, framing energy consumption as a central constraint alongside memory and compute. It surveys design-time optimizations, energy-adaptive inference (including DNN right-sizing, multi-exit networks, and offloading), on-device training, relevant applications (notably energy-harvesting devices and energy-aware Federated Learning), and hardware trends. It highlights limitations in current energy-estimation methods and the reliance on proxies like MACs, emphasizing the need for cross-platform energy prediction and automated profiling. The work underlines practical implications for deploying DL on severely energy-constrained hardware and outlines future directions to enable robust, energy-efficient AI at the edge. The insights aim to drive co-design of hardware accelerators, memory-centric architectures, and energy-aware training/inference strategies that sustain performance under intermittent power and limited resources.

Abstract

The use of deep learning (DL) on Internet of Things (IoT) and mobile devices offers numerous advantages over cloud-based processing. However, such devices face substantial energy constraints to prolong battery-life, or may even operate intermittently via energy-harvesting. Consequently, \textit{energy-aware} approaches for optimizing DL inference and training on such resource-constrained devices have garnered recent interest. We present an overview of such approaches, outlining their methodologies, implications for energy consumption and system-level efficiency, and their limitations in terms of supported network types, hardware platforms, and application scenarios. We hope our review offers a clear synthesis of the evolving energy-aware DL landscape and serves as a foundation for future research in energy-constrained computing.

Energy-Aware Deep Learning on Resource-Constrained Hardware

TL;DR

This survey analyzes energy-aware deep learning for IoT and edge devices, framing energy consumption as a central constraint alongside memory and compute. It surveys design-time optimizations, energy-adaptive inference (including DNN right-sizing, multi-exit networks, and offloading), on-device training, relevant applications (notably energy-harvesting devices and energy-aware Federated Learning), and hardware trends. It highlights limitations in current energy-estimation methods and the reliance on proxies like MACs, emphasizing the need for cross-platform energy prediction and automated profiling. The work underlines practical implications for deploying DL on severely energy-constrained hardware and outlines future directions to enable robust, energy-efficient AI at the edge. The insights aim to drive co-design of hardware accelerators, memory-centric architectures, and energy-aware training/inference strategies that sustain performance under intermittent power and limited resources.

Abstract

The use of deep learning (DL) on Internet of Things (IoT) and mobile devices offers numerous advantages over cloud-based processing. However, such devices face substantial energy constraints to prolong battery-life, or may even operate intermittently via energy-harvesting. Consequently, \textit{energy-aware} approaches for optimizing DL inference and training on such resource-constrained devices have garnered recent interest. We present an overview of such approaches, outlining their methodologies, implications for energy consumption and system-level efficiency, and their limitations in terms of supported network types, hardware platforms, and application scenarios. We hope our review offers a clear synthesis of the evolving energy-aware DL landscape and serves as a foundation for future research in energy-constrained computing.
Paper Structure (28 sections, 2 equations, 3 figures, 3 tables)

This paper contains 28 sections, 2 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works and Contributions
  3. Structure
  4. Energy-Aware DNN Design
  5. Design-Time Optimizations
  6. Neural Architecture Search
  7. Predicting Inference Energy
  8. Energy-Adaptive Inference
  9. DNN Right-Sizing
  10. Multi-Exit Networks
  11. Multi-Exit Design
  12. Deployment of Multi-Exit Networks
  13. Energy-Adaptive Inference Policies
  14. Inference Offloading
  15. On-Device Training
  16. ...and 13 more sections

Figures (3)

  • Figure 1: Conventional DNN architecture alongside an EE variant. The dashed lines indicate conditional execution paths.
  • Figure 2: Multi-exit last-layer distillation training, with a backbone network trained in the cloud and its exits trained on-device.
  • Figure 3: DNN partitioning between device and server.