An approach based on class activation maps for investigating the effects of data augmentation on neural networks for image classification

TL;DR

This work tackles how data augmentation reshapes the pixel-level attribution patterns learned by CNNs in image classification. It introduces a scalable methodology that uses Grad-CAM to generate Class Activation Maps from a baseline model and multiple augmented models, comparing them with a suite of metrics (MAD, MSD, Pearson and Spearman correlations, Overlap Rate, and Class-KLD) across $|A|$ augmentations and $|X|$ seeds. Applying this to CIFAR-10 with EfficientNet-B0 demonstrates that augmentations can improve performance while altering attribution patterns in measurable but not always easily separable ways; results suggest distinct augmentation profiles and partial clustering of augmentation effects. The study provides a structured, extensible framework for quantitatively analyzing augmentation impact via CAMs, with implications for designing robust augmentation strategies and guiding explainability-focused model diagnostics.

Abstract

Neural networks have become increasingly popular in the last few years as an effective tool for the task of image classification due to the impressive performance they have achieved on this task. In image classification tasks, it is common to use data augmentation strategies to increase the robustness of trained networks to changes in the input images and to avoid overfitting. Although data augmentation is a widely adopted technique, the literature lacks a body of research analyzing the effects data augmentation methods have on the patterns learned by neural network models working on complex datasets. The primary objective of this work is to propose a methodology and set of metrics that may allow a quantitative approach to analyzing the effects of data augmentation in convolutional networks applied to image classification. An important tool used in the proposed approach lies in the concept of class activation maps for said models, which allow us to identify and measure the importance these models assign to each individual pixel in an image when executing the classification task. From these maps, we may then extract metrics over the similarities and differences between maps generated by these models trained on a given dataset with different data augmentation strategies. Experiments made using this methodology suggest that the effects of these data augmentation techniques not only can be analyzed in this way but also allow us to identify different impact profiles over the trained models.

Figures (49)

• Figure 8: Example of CAM generated by Grad-CAM using the baseline model we trained as the selected model, the last convolutional layer as the target layer, and the predicted class as the target class.
• Figure 9: Evolution of the accuracy on the test set across 30 training epochs for the three candidate architectures.
• Figure 10: Evolution of the precision on the test set across 30 training epochs for the three candidate architectures.
• Figure 11: Evolution of the recall on the test set across 30 training epochs for the three candidate architectures.
• Figure 12: Evolution of the F1-score on the test set across 30 training epochs for the three candidate architectures.