Wider or Deeper: Revisiting the ResNet Model for Visual Recognition

TL;DR

This work challenges the notion that deeper nets inherently yield better performance, proposing an unravelled view and the concept of effective depth to reinterpret ResNets. It argues that ResNets function as linearly growing ensembles of sub-networks, and that fully end-to-end trainable, shallower architectures can match or exceed the performance of very deep models. The authors design a family of shallower, wide residual networks that achieve state-of-the-art results on ImageNet and translate these gains to semantic segmentation across VOC, Cityscapes, ADE20K, and Pascal Context. The results demonstrate improved efficiency (memory and often training time) and robust transfer to dense prediction tasks, with code and models publicly available.

Abstract

The trend towards increasingly deep neural networks has been driven by a general observation that increasing depth increases the performance of a network. Recently, however, evidence has been amassing that simply increasing depth may not be the best way to increase performance, particularly given other limitations. Investigations into deep residual networks have also suggested that they may not in fact be operating as a single deep network, but rather as an ensemble of many relatively shallow networks. We examine these issues, and in doing so arrive at a new interpretation of the unravelled view of deep residual networks which explains some of the behaviours that have been observed experimentally. As a result, we are able to derive a new, shallower, architecture of residual networks which significantly outperforms much deeper models such as ResNet-200 on the ImageNet classification dataset. We also show that this performance is transferable to other problem domains by developing a semantic segmentation approach which outperforms the state-of-the-art by a remarkable margin on datasets including PASCAL VOC, PASCAL Context, and Cityscapes. The architecture that we propose thus outperforms its comparators, including very deep ResNets, and yet is more efficient in memory use and sometimes also in training time. The code and models are available at https://github.com/itijyou/ademxapp

Figures (11)

• Figure 1: The unravelled view of a simple ResNet. The fact that $f_2(\cdot)$ is non-linear gives rise to the inequality in the top row, as $f_2(a+b)\neq f_2(a)+ f_2(b)$. This shows that $f_2(\cdot)$ never operates independently of the result of $f_1(\cdot)$, and thus that the number of independent classifiers increases linearly with the number of residual units. Analysing the interactions between residual units at a given effective depth, here labelled $l$, illuminates the paths taken by gradients during training.
• Figure 2: The impact of inserting an extra residual unit into a two-unit ResNet, which depends on the effective depth $l$.
• Figure 3: Overview of our proposed networks with different input sizes. Note that B1--B7 are respectively a residual unit.
• Figure 4: Gradient magnitude at input given a path length $k$ in various residual networks. See the text for details.
• Figure 5: Qualitative results on the PASCAL VOC 2012 PascalVoc.IJCV.2014.Everingham val set. The model was trained using the train set augmented using SBD SBD.ICCV.2011.Hariharan. In each example, from top to bottom, there are in turn the original image, the ground-truth, the predicted label, and the difference map between the ground-truth and the predicted label.