Unsupervised learning based object detection using Contrastive Learning

TL;DR

The paper tackles the challenge of unsupervised object detection by introducing a two-branch contrastive framework that learns both appearance and location information. It combines inter-image and intra-image contrastive learning within an anchor-based NT-Xent loss to produce location-aware embeddings and heatmaps, trained end-to-end on COCO without labels. The method achieves an impressive 89.2% Similarity Grid Accuracy, vastly outperforming random initialization, and demonstrates potential to substantially reduce labeling costs while enabling robust localization on diverse, unlabeled data. This approach thus offers a practical path toward scalable, unsupervised object detection with interpretable heatmaps for localization.

Abstract

Training image-based object detectors presents formidable challenges, as it entails not only the complexities of object detection but also the added intricacies of precisely localizing objects within potentially diverse and noisy environments. However, the collection of imagery itself can often be straightforward; for instance, cameras mounted in vehicles can effortlessly capture vast amounts of data in various real-world scenarios. In light of this, we introduce a groundbreaking method for training single-stage object detectors through unsupervised/self-supervised learning. Our state-of-the-art approach has the potential to revolutionize the labeling process, substantially reducing the time and cost associated with manual annotation. Furthermore, it paves the way for previously unattainable research opportunities, particularly for large, diverse, and challenging datasets lacking extensive labels. In contrast to prevalent unsupervised learning methods that primarily target classification tasks, our approach takes on the unique challenge of object detection. We pioneer the concept of intra-image contrastive learning alongside inter-image counterparts, enabling the acquisition of crucial location information essential for object detection. The method adeptly learns and represents this location information, yielding informative heatmaps. Our results showcase an outstanding accuracy of \textbf{89.2\%}, marking a significant breakthrough of approximately \textbf{15x} over random initialization in the realm of unsupervised object detection within the field of computer vision.

Figures (5)

• Figure 1: The diagram illustrates the interplay between our two pipelines. In the upper pipeline, refered to as Pipeline 1, we begin with input data $x_i$ and proceed to process the image, ultimately generating a representation suitable for deployment in our Anchor-Based NT-Xent loss. Similarly, the lower pipeline, referred to as Pipeline 2, takes the input $x_j$ and conducts image processing operations, culminating in the extraction of FPN outputs. These FPN outputs are thoughtfully curated to identify positive and negative samples within the image, as depicted below.
• Figure 2: Selection of embeddings from FPN layers
• Figure 3: Augmentations
• Figure 4: In this visual representation, we present the model's output per layer. Our approach involves showcasing the representations obtained at each layer of the Feature Pyramid Network (FPN) and distributing the corresponding similarities, centered around the FPN grid cells, across the entire image. This approach enables the generation of heatmaps for each layer of the FPN, providing valuable insights into the model's hierarchical feature representations.
• Figure 5: This figure shows a grid of images gathered after selecting a crop within the dataset and searching the top10 similar images. The selected crop is passed to the RetinaNet to produce a representation and the highest similarity images are process on a batch to produce the FPN outputs used to compare and execute the selection.