Early-Exit with Class Exclusion for Efficient Inference of Neural Networks

TL;DR

The paper tackles the high computational burden of deep neural networks on edge devices by proposing a class-based early-exit mechanism that proactively excludes irrelevant classes using features from early layers. It introduces per-class exclusion networks with global average pooling and Sigmoid activations, coupled with a class-exclusion aware training objective and a dynamic inference policy that thresholds class exclusion using a beta parameter. The method demonstrates substantial reductions in FLOPs/MACs while maintaining competitive accuracy across CIFAR-10/100 with AlexNet, VGGsmall, and ResNet50, and it outperforms prior confidence-based early-exit approaches in remaining computation. The work contributes a practical framework for efficient inference and provides open-source code to support reproduction and adoption in edge scenarios.

Abstract

Deep neural networks (DNNs) have been successfully applied in various fields. In DNNs, a large number of multiply-accumulate (MAC) operations are required to be performed, posing critical challenges in applying them in resource-constrained platforms, e.g., edge devices. To address this challenge, in this paper, we propose a class-based early-exit for dynamic inference. Instead of pushing DNNs to make a dynamic decision at intermediate layers, we take advantage of the learned features in these layers to exclude as many irrelevant classes as possible, so that later layers only have to determine the target class among the remaining classes. When only one class remains at a layer, this class is the corresponding classification result. Experimental results demonstrate the computational cost of DNNs in inference can be reduced significantly with the proposed early-exit technique. The codes can be found at https://github.com/HWAI-TUDa/EarlyClassExclusion.

Figures (8)

• Figure 1: (a) Traditional early-exit strategy. (b) The proposed class-exclusion early-exit strategy.
• Figure 2: Construction of class-exclusion neural networks for four classes.
• Figure 3: Class exclusion based on a dynamic threshold.
• Figure 4: The number of input images that can be classified in the intermediate exit point of neural networks.
• Figure 5: Percentage of classes that can exit early with respect to the exited images. For CIFAR10, class 0-9 are airplane, automobile, bird, cat, deer, dog, frog, horse, ship and truck, respectively.