Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Study

TL;DR

This paper investigates post-hoc pruning as a practical compression strategy for CNNs and its impact on safety-relevant properties, namely uncertainty calibration and natural corruption robustness. It benchmarks three static post-hoc pruning methods (unstructured weight pruning, structured filter pruning, and structured channel pruning) on CIFAR-10 and CIFAR-10-C with VGG and ResNet backbones. Across pruning levels, the study finds that post-hoc pruning consistently improves or preserves uncertainty calibration and robustness to natural corruptions, with unstructured pruning offering the strongest calibration gains and structured pruning maintaining calibration up to moderate compression. The findings suggest that accuracy-driven pruning does not degrade, and can even enhance, safety properties, supporting safer deployment of compressed CNNs in embedded and safety-critical applications.

Abstract

Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in many computer vision tasks. However, high computational and storage demands hinder their deployment into resource-constrained environments, such as embedded devices. Model pruning helps to meet these restrictions by reducing the model size, while maintaining superior performance. Meanwhile, safety-critical applications pose more than just resource and performance constraints. In particular, predictions must not be overly confident, i.e., provide properly calibrated uncertainty estimations (proper uncertainty calibration), and CNNs must be robust against corruptions like naturally occurring input perturbations (natural corruption robustness). This work investigates the important trade-off between uncertainty calibration, natural corruption robustness, and performance for current state-of-research post-hoc CNN pruning techniques in the context of image classification tasks. Our study reveals that post-hoc pruning substantially improves the model's uncertainty calibration, performance, and natural corruption robustness, sparking hope for safe and robust embedded CNNs.Furthermore, uncertainty calibration and natural corruption robustness are not mutually exclusive targets under pruning, as evidenced by the improved safety aspects obtained by post-hoc unstructured pruning with increasing compression.

Figures (3)

• Figure 1: Comparing development of $\mathit{ECE}$ ($\downarrow$) (blue lines, left y-axis scaling) and accuracy ($\uparrow$) (orange lines, right y-axis scaling) under increasing pruning ratios
• Figure 2: Development of $\mathit{mPC}$ ($\uparrow$) for increasing pruning ratios for five severity levels of corruption
• Figure 3: Development of $\mathit{ECE}$ ($\downarrow$) for increasing pruning ratio for five severity levels of corruption