Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques

TL;DR

This survey addresses the challenge of achieving resource-efficient deep learning across devices by organizing techniques into model-, arithmetic-, and implementation-level categories and by defining multi-dimensional resource-efficiency metrics. It covers model-level methods (quantization, pruning, compact convolutions, distillation, NAS) and arithmetic-/implementation-level strategies (precision formats, mixed precision, data reuse, sparsity, and hardware-aware dataflow), with concrete mechanisms and trade-offs. A key contribution is clarifying how higher-level design choices influence lower-level efficiency and proposing an integrated perspective for jointly optimizing accuracy, memory, compute, and energy on diverse hardware, with a baseline metric framework including $P_D = #_{TTR} \times C_{CP} \times V_{CP}^2 \times f_{CP}$ for power. The paper also outlines promising directions—domain-aware NAS, adaptive arithmetic formats, edge/distributed AI, and neuromorphic/in-memory computing—to push resource efficiency toward real-world deployment.

Abstract

Deep learning is pervasive in our daily life, including self-driving cars, virtual assistants, social network services, healthcare services, face recognition, etc. However, deep neural networks demand substantial compute resources during training and inference. The machine learning community has mainly focused on model-level optimizations such as architectural compression of deep learning models, while the system community has focused on implementation-level optimization. In between, various arithmetic-level optimization techniques have been proposed in the arithmetic community. This article provides a survey on resource-efficient deep learning techniques in terms of model-, arithmetic-, and implementation-level techniques and identifies the research gaps for resource-efficient deep learning techniques across the three different level techniques. Our survey clarifies the influence from higher to lower-level techniques based on our resource-efficiency metric definition and discusses the future trend for resource-efficient deep learning research.

Figures (9)

• Figure 1: Survey on resource-efficient deep learning techniques based on resource efficiency metrics.
• Figure 2: Perceptron and neural network model.
• Figure 3: Convolution operations in a CNN.
• Figure 4: Categorization for model-level resource-efficient techniques.
• Figure 5: Depth-wise convolution used in howard-mobilenets.