Deep Pyramidal Residual Networks with Separated Stochastic Depth

TL;DR

Deep Pyramidal Residual Networks with Separated Stochastic Depth addresses the challenge of training very deep CNNs by combining ResDrop's stochastic depth with PyramidNet's gradual channel growth. The authors introduce PyramidSepDrop, a separated residual design where the residual function is split into an upper, channel-increasing path and a lower, input-like path, both subjected to stochastic depth. On CIFAR-100, PyramidSepDrop achieves a 16.18% error, outperforming PyramidNet's 18.29% and ResNeXt's 17.31%, demonstrating improved optimization and accuracy. This approach offers a practical pathway to more effective deep residual architectures by harmonizing depth, channel growth, and stochastic regularization.

Abstract

On general object recognition, Deep Convolutional Neural Networks (DCNNs) achieve high accuracy. In particular, ResNet and its improvements have broken the lowest error rate records. In this paper, we propose a method to successfully combine two ResNet improvements, ResDrop and PyramidNet. We confirmed that the proposed network outperformed the conventional methods; on CIFAR-100, the proposed network achieved an error rate of 16.18% in contrast to PiramidNet achieving that of 18.29% and ResNeXt 17.31%.

Figures (3)

• Figure 1: Residual block of PyramidDrop
• Figure 2: Residual block of PyramidSepDrop
• Figure 3: Error rates of PyramidSepDrop on the last epoch when increasing the number of models used for multi-model learning.