Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating the Impact of Compression Techniques on the Robustness of CNNs under Natural Corruptions

Itallo Patrick Castro Alves Da Silva, Emanuel Adler Medeiros Pereira, Erick de Andrade Barboza, Baldoino Fonseca dos Santos Neto, Marcio de Medeiros Ribeiro

TL;DR

This work addresses the robustness of convolutional neural networks under natural corruptions when deployed with compression for edge devices. It rigorously evaluates quantization, pruning, and weight sharing—individually and in combinations—on ResNet-50, VGG-19, and MobileNetV2 using CIFAR-10-C and CIFAR-100-C, starting from ImageNet-pretrained weights and applying transfer learning. Robustness, accuracy, and compression efficiency are measured via $mCE$, accuracy, and the compression ratio, with Pareto-front analysis to identify balanced configurations. The findings show that certain combined compression strategies can preserve or even improve robustness, especially for larger architectures, while some settings (e.g., QAT with Int8) may underperform; MobileNetV2 often lies outside the Pareto-optimal set. The results provide practical guidance for selecting compression pipelines that maintain robustness in corrupted real-world environments and highlight the value of multiobjective evaluation for edge deployments.

Abstract

Compressed deep learning models are crucial for deploying computer vision systems on resource-constrained devices. However, model compression may affect robustness, especially under natural corruption. Therefore, it is important to consider robustness evaluation while validating computer vision systems. This paper presents a comprehensive evaluation of compression techniques - quantization, pruning, and weight clustering applied individually and in combination to convolutional neural networks (ResNet-50, VGG-19, and MobileNetV2). Using the CIFAR-10-C and CIFAR 100-C datasets, we analyze the trade-offs between robustness, accuracy, and compression ratio. Our results show that certain compression strategies not only preserve but can also improve robustness, particularly on networks with more complex architectures. Utilizing multiobjective assessment, we determine the best configurations, showing that customized technique combinations produce beneficial multi-objective results. This study provides insights into selecting compression methods for robust and efficient deployment of models in corrupted real-world environments.

Evaluating the Impact of Compression Techniques on the Robustness of CNNs under Natural Corruptions

TL;DR

This work addresses the robustness of convolutional neural networks under natural corruptions when deployed with compression for edge devices. It rigorously evaluates quantization, pruning, and weight sharing—individually and in combinations—on ResNet-50, VGG-19, and MobileNetV2 using CIFAR-10-C and CIFAR-100-C, starting from ImageNet-pretrained weights and applying transfer learning. Robustness, accuracy, and compression efficiency are measured via , accuracy, and the compression ratio, with Pareto-front analysis to identify balanced configurations. The findings show that certain combined compression strategies can preserve or even improve robustness, especially for larger architectures, while some settings (e.g., QAT with Int8) may underperform; MobileNetV2 often lies outside the Pareto-optimal set. The results provide practical guidance for selecting compression pipelines that maintain robustness in corrupted real-world environments and highlight the value of multiobjective evaluation for edge deployments.

Abstract

Compressed deep learning models are crucial for deploying computer vision systems on resource-constrained devices. However, model compression may affect robustness, especially under natural corruption. Therefore, it is important to consider robustness evaluation while validating computer vision systems. This paper presents a comprehensive evaluation of compression techniques - quantization, pruning, and weight clustering applied individually and in combination to convolutional neural networks (ResNet-50, VGG-19, and MobileNetV2). Using the CIFAR-10-C and CIFAR 100-C datasets, we analyze the trade-offs between robustness, accuracy, and compression ratio. Our results show that certain compression strategies not only preserve but can also improve robustness, particularly on networks with more complex architectures. Utilizing multiobjective assessment, we determine the best configurations, showing that customized technique combinations produce beneficial multi-objective results. This study provides insights into selecting compression methods for robust and efficient deployment of models in corrupted real-world environments.
Paper Structure (5 sections, 1 equation, 3 figures, 6 tables)

This paper contains 5 sections, 1 equation, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Results and discussion
  5. Conclusion and future works

Figures (3)

  • Figure 1: The 15 types of corruption considered in this work (Source: hendrycks2018benchmarking).
  • Figure 2: Tree with Tensorflow's collaborative optimizations (Source: tensorflow2015-whitepaper).
  • Figure 3: Graphical representation of the optimization results for (a) CIFAR-10 and (b) CIFAR-100, considering the multi-objective evaluation of mCE (mean Corruption Error), compression rate, and accuracy. Each point corresponds to a specific model optimization technique, as detailed in Table \ref{['tab:applied_techniques']}. The color gradient represents the accuracy, while the symbols distinguish different model architectures. The dashed line highlights the Pareto front, and the solid line denotes the baseline performance of the original models.