Taming the Randomness: Towards Label-Preserving Cropping in Contrastive Learning

TL;DR

This paper tackles false labeling in contrastive self-supervised learning caused by random cropping by introducing parameterized Gaussian-centered cropping (GCC) and Multi-Object Gaussian-Centered Cropping (MGCC). These methods sample crop regions with controllable variance and, in MGCC, a mean around the center to accommodate multiple objects, improving the quality of positive pairs without heavy overhead. Across CIFAR-10, TinyImageNet, and ImageNet64, GCC and MGCC outperform RandomCrop, with MGCC showing particular strength on multi-object datasets. The approach offers a simple yet effective augmentation strategy that enhances downstream linear classification performance, highlighting practical gains for self-supervised CV tasks.

Abstract

Contrastive learning (CL) approaches have gained great recognition as a very successful subset of self-supervised learning (SSL) methods. SSL enables learning from unlabeled data, a crucial step in the advancement of deep learning, particularly in computer vision (CV), given the plethora of unlabeled image data. CL works by comparing different random augmentations (e.g., different crops) of the same image, thus achieving self-labeling. Nevertheless, randomly augmenting images and especially random cropping can result in an image that is semantically very distant from the original and therefore leads to false labeling, hence undermining the efficacy of the methods. In this research, two novel parameterized cropping methods are introduced that increase the robustness of self-labeling and consequently increase the efficacy. The results show that the use of these methods significantly improves the accuracy of the model by between 2.7\% and 12.4\% on the downstream task of classifying CIFAR-10, depending on the crop size compared to that of the non-parameterized random cropping method.

Figures (6)

• Figure 1: Effect of the variance scaling parameter alpha on the performance of each of the proposed methods. Subplots show different crop sizes. The pretraining was performed on CIFAR-10 for 200 epochs.
• Figure 2: Methods performance regarding crop size. Showing the accuracy of the downstream classifier after pretraining on CIFAR-10 for 500 epochs with varying crop size.
• Figure 3: Methods performance regarding crop size. Showing the accuracy of the downstream classifier after pretraining on TinyImageNet for 200 epochs with varying crop size.
• Figure 4: Methods performance regarding crop size. Showing the accuracy of the downstream classifier after pretraining on ImageNet64 for 200 epochs with varying crop size.
• Figure 5: The effect of the different cropping methods applied to a centered image with 20% crop size. The crops are ordered from left to right each consecutive pair of images in a row represents a proposed positive pair, the red boxes indicate this pairwise relationship.