Early-exit Convolutional Neural Networks

TL;DR

Through comprehensive experiments on MNIST, SVHN, CIFAR10 and Tiny-ImageNet datasets, it is shown that early-exit ResNets achieve similar accuracy with their non-EE versions while reducing the computational cost to 20% of the original.

Abstract

This paper is aimed at developing a method that reduces the computational cost of convolutional neural networks (CNN) during inference. Conventionally, the input data pass through a fixed neural network architecture. However, easy examples can be classified at early stages of processing and conventional networks do not take this into account. In this paper, we introduce 'Early-exit CNNs', EENets for short, which adapt their computational cost based on the input by stopping the inference process at certain exit locations. In EENets, there are a number of exit blocks each of which consists of a confidence branch and a softmax branch. The confidence branch computes the confidence score of exiting (i.e. stopping the inference process) at that location; while the softmax branch outputs a classification probability vector. Both branches are learnable and their parameters are separate. During training of EENets, in addition to the classical classification loss, the computational cost of inference is taken into account as well. As a result, the network adapts its many confidence branches to the inputs so that less computation is spent for easy examples. Inference works as in conventional feed-forward networks, however, when the output of a confidence branch is larger than a certain threshold, the inference stops for that specific example. The idea of EENets is applicable to available CNN architectures such as ResNets. Through comprehensive experiments on MNIST, SVHN, CIFAR10 and Tiny-ImageNet datasets, we show that early-exit (EE) ResNets achieve similar accuracy with their non-EE versions while reducing the computational cost to 20% of the original. Code is available at https://github.com/eksuas/eenets.pytorch

Figures (8)

• Figure 1: Architectural overview of EENets. An early-exit block (shown with gray color) can be added at any location. If, at a certain early-exit block, say the $i^{th}$ one, the network is sufficiently confident (i.e. $h_i>0.5$), then the execution is terminated at that point and the network's output is set to $\hat{y}_i$. $c_i$ denotes the computational cost (in terms of the total number of floating-point operations) upto the $i^{th}$ early-exit block. "CNN layers" are classical computation blocks that may be composed of one or more convolutional or fully-connected layers and non-linear activation functions. EENets aim to strike a balance between minimizing the computational cost and maximizing the accuracy.
• Figure 2: Architecture of plain, Pool, and Bnpool early-exit blocks. The plain-type exits are composed of just separate fully-connected (FC) layers and input of that block is directly processed in the FC branches. The Pool exits have a global average pooling layer before FC branches. Lastly, the Bnpool-type exit blocks consist of a batch normalization layer followed by a ReLU activation and a global average pooling layer. The input of the early-exit block, $x$, passes onto these layers before entering the separate FC branches. $h$ and $\hat{y}_{n}$ denote the confidence score and the predicted classification label. “$\mathrm{fc}$$X, \mathrm{activation}$” denotes the fully-connected heads which have $X$ number of outputs. The $\mathrm{activation}$ is the last activation function of branches.
• Figure 3: Distributing early-exit blocks to a network. Pareto, Golden Ratio and Fine can be represented in the upper figure. The $\varphi$ denotes the ratio used in the methods. For example, $\varphi$ will be $0.2$, $0.6180$ and $0.05$ for Pareto, Golden Ratio and Fine distributions, respectively. $N$ shows the number of early-exit blocks. The below figure shows the Linear distribution where the computational costs between consecutive early-exit blocks are same and this cost can be calculated by the desired number of the early-exit blocks. Notice that the total cost is represented by 1 since our cost terms are rates (i.e. $c_i \in [0, 1]$).
• Figure 4: EENet-101 model with three early-exit blocks. The Pool-type of early-exit blocks and the bottleneck blocks are employed. The architecture is in the form of Naive ResNet models.
• Figure 5: EENet-110 model with two early-exit blocks. The early-exit blocks are in the pool-type. It is another example of the architectures in the 6n+2 ResNet form.