Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

[Re] Network Deconvolution

Rochana R. Obadage, Kumushini Thennakoon, Sarah M. Rajtmajer, Jian Wu

TL;DR

This effort confirms the validity of the authors' claim that using network deconvolution instead of batch normalization improves deep learning model performance, and reproduces the results reported in Tables 1 and 2 of the original paper.

Abstract

Our work aims to reproduce the set of findings published in "Network Deconvolution" by Ye et al. (2020)[1]. That paper proposes an optimization technique for model training in convolutional neural networks. The proposed technique "network deconvolution" is used in convolutional neural networks to remove pixel-wise and channel-wise correlations before data is fed into each layer. In particular, we interrogate the validity of the authors' claim that using network deconvolution instead of batch normalization improves deep learning model performance. Our effort confirms the validity of this claim, successfully reproducing the results reported in Tables 1 and 2 of the original paper. Our study involved 367 unique experiments across multiple architectures, datasets, and hyper parameter configurations. For Table 1, while there were some minor deviations in accuracy when compared to the original values (within 10%), the overall trend was consistent with the original study's findings when training the models with epochs 20 and 100. For Table 2, all 14 reproduced values were consistent with the original values. Additionally, we document the training and testing times for each architecture in Table 1 with 1, 20, and 100 epoch settings for both CIFAR-10 and CIFAR-100 datasets. We document the total execution times for Table 2 architectures with the ImageNet dataset. The data and software used for this reproducibility study are publicly available at https://github.com/lamps-lab/rep-network-deconvolution.

[Re] Network Deconvolution

TL;DR

This effort confirms the validity of the authors' claim that using network deconvolution instead of batch normalization improves deep learning model performance, and reproduces the results reported in Tables 1 and 2 of the original paper.

Abstract

Our work aims to reproduce the set of findings published in "Network Deconvolution" by Ye et al. (2020)[1]. That paper proposes an optimization technique for model training in convolutional neural networks. The proposed technique "network deconvolution" is used in convolutional neural networks to remove pixel-wise and channel-wise correlations before data is fed into each layer. In particular, we interrogate the validity of the authors' claim that using network deconvolution instead of batch normalization improves deep learning model performance. Our effort confirms the validity of this claim, successfully reproducing the results reported in Tables 1 and 2 of the original paper. Our study involved 367 unique experiments across multiple architectures, datasets, and hyper parameter configurations. For Table 1, while there were some minor deviations in accuracy when compared to the original values (within 10%), the overall trend was consistent with the original study's findings when training the models with epochs 20 and 100. For Table 2, all 14 reproduced values were consistent with the original values. Additionally, we document the training and testing times for each architecture in Table 1 with 1, 20, and 100 epoch settings for both CIFAR-10 and CIFAR-100 datasets. We document the total execution times for Table 2 architectures with the ImageNet dataset. The data and software used for this reproducibility study are publicly available at https://github.com/lamps-lab/rep-network-deconvolution.
Paper Structure (21 sections, 5 figures, 10 tables)

This paper contains 21 sections, 5 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Citations of the Original Study from Scholarly Papers
  3. Scope of reproducibility
  4. Methodology
  5. Datasets
  6. Hyperparameters
  7. Preparing the ImageNet dataset
  8. Experimental setup
  9. Computational requirements
  10. Recommendations for Minimal Computational Requirements
  11. Results
  12. Reproduced Results of Table 1 in the Original paper
  13. Reproduced Results of Table 2 in the Original Paper
  14. Training Time Comparison
  15. Discussion
  16. ...and 6 more sections

Figures (5)

  • Figure 1: Network deconvolution. Adopted from Figure 3 and Algorithm 1 in the original study ye2020network.
  • Figure 2: Comparing original and reproduced accuracy for CIFAR-10 dataset: Original Study ye2020network - Table 1 (a) with 1 epoch, (b) with 20 epochs, (c) with 100 epochs. (Legend: O_BN_20 - original accuracy using batch normalization with 20 epochs, R_ND_100 - reproduced accuracy using network deconvolution with 100 epochs).
  • Figure 3: Comparing original and reproduced accuracy for CIFAR-100 dataset: Original Study ye2020network - Table 1 (a) with 1 epoch, (b) with 20 epochs, (c) with 100 epochs. Labels: (Legend: O_BN_20 - original accuracy using batch normalization with 20 epochs, R_ND_100 - reproduced accuracy using network deconvolution with 100 epochs).
  • Figure 4: Training times for each CNN architecture with CIFAR-10 dataset: (a) with 1 epoch, (b) with 20 epochs, (c) with 100 epochs
  • Figure 5: Training times for each CNN architecture with CIFAR-100 dataset: (a) with 1 epoch, (b) with 20 epochs, (c) with 100 epochs