On the Relationship Between Double Descent of CNNs and Shape/Texture Bias Under Learning Process

TL;DR

This work investigates the relationship between epoch-wise double descent in CNNs and learning of shape versus texture features in natural images. It combines a phase-based framework for tracking test error with Islam's neuron-based bias quantification to examine how shape/texture bias evolves during training under double-descent conditions, including Nakkiran's CIFAR-10 setup and various ablations. The study finds that shape bias often correlates positively with test error during Phases 1-2 while texture bias correlates negatively, with inflection points aligning closely to error changes; double descent/ascent of bias can occur even without label noise, and deeper layers show stronger synchronization with test performance. These results provide mechanistic insight into the learning dynamics underlying double descent in CNNs and suggest bias trajectories could inform training strategies and bias-aware modeling for image recognition tasks.

Abstract

The double descent phenomenon, which deviates from the traditional bias-variance trade-off theory, attracts considerable research attention; however, the mechanism of its occurrence is not fully understood. On the other hand, in the study of convolutional neural networks (CNNs) for image recognition, methods are proposed to quantify the bias on shape features versus texture features in images, determining which features the CNN focuses on more. In this work, we hypothesize that there is a relationship between the shape/texture bias in the learning process of CNNs and epoch-wise double descent, and we conduct verification. As a result, we discover double descent/ascent of shape/texture bias synchronized with double descent of test error under conditions where epoch-wise double descent is observed. Quantitative evaluations confirm this correlation between the test errors and the bias values from the initial decrease to the full increase in test error. Interestingly, double descent/ascent of shape/texture bias is observed in some cases even in conditions without label noise, where double descent is thought not to occur. These experimental results are considered to contribute to the understanding of the mechanisms behind the double descent phenomenon and the learning process of CNNs in image recognition.

Figures (9)

• Figure 1: Flow of analysis process presented in this paper. We train CNNs for image recognition under double descent conditions. We monitor the temporal evolution of the shape/texture bias and test error assessing the capacity of the model to interpret shapes and textures while exploring their correlation.
• Figure 2: Schematic overview of this study. Top left: The learning curve of the CIFAR-10 image recognition task under the setting of nakkiran2021deep et al, where epoch-wise double descent was observed. Test errors were divided into three phases based on their temporal differentiation. Bottom left: This records the model's shape/texture bias during the aforementioned learning process. It shows the synchronous changes between test errors and shape/texture bias. Right: A scatter plot of test error and shape/texture bias. Especially in Phase 1 and Phase 2, there is a positive correlation between test error and shape bias, and a negative correlation between test error and texture bias. In all bias visualization settings, including this one, we use a 5-term moving average to smooth the data for trend analysis.
• Figure 3: Overview of Islam's method. This figure shows an overview of the process of calculating the shape and texture bias using the method of Islam et al. Islam
• Figure 4: Comparison of training with randomly initialized weights (Scratch) and with weights pre-trained on ImageNet (ImageNet). Left top: train and test errors (%). Left bottom: shape/texture bias (%). Right: Enlarged view.
• Figure 5: Comparison of CIFAR-10 and CIFAR-100. Top: train and test errors (%). Bottom: shape/texture bias (%).