Dynamic LRP-Based Pruning for CNNs in Data-Scarce Transfer Learning: Suppressing Cascading Accuracy Degradation
Daisuke Yasui, Toshitaka Matsuki, Hiroshi Sato
TL;DR
This paper tackles pruning of ImageNet-pretrained CNNs in data-scarce transfer learning, where retraining is risky and redundancy hampers efficiency. It identifies cascading accuracy degradation caused by imbalanced class performance when pruning based on Layer-wise Relevance Propagation (LRP) and introduces DPX-SD, which dynamically adapts pruning rate and order to preserve task-specific accuracy. Two mechanisms—Change of pruning rate and Change of pruning order—mitigate the degradation by maintaining a harmonic-mean class accuracy $A$ and re-evaluating relevance as pruning proceeds. Across four architectures and constrained data, DPX-SD achieves higher accuracy at greater pruning levels than prior LRP-based methods, albeit with higher offline computation, highlighting its potential for practical deployment in resource-limited settings where model compression is essential.
Abstract
Convolutional Neural Networks (CNNs) pre-trained on large-scale datasets such as ImageNet are widely used as feature extractors to construct high-accuracy classification models from scarce data for specific tasks. In such scenarios, fine-tuning the pre-trained CNN is difficult due to data scarcity, necessitating the use of fixed weights. However, when the weights are kept fixed, many filters that do not contribute to the target task remain in the model, leading to unnecessary redundancy and reduced efficiency. Therefore, effective methods are needed to reduce model size by pruning filters that are unnecessary for inference. To address this, approaches utilizing Layer-wise Relevance Propagation (LRP) have been proposed. LRP quantifies the contribution of each filter to the inference result, enabling the pruning of filters with low relevance. However, existing LRP-based pruning methods have been observed to cause cascading accuracy degradation. In this study, we propose an LRP-based dynamic pruning method that suppresses this cascading accuracy degradation and compresses the pre-trained model while preserving task-specific performance in a small-data environment. We demonstrate that the proposed method effectively mitigates the cascading accuracy degradation and achieves higher classification accuracy compared to existing LRP-based pruning methods.