Concept Drift and Long-Tailed Distribution in Fine-Grained Visual Categorization: Benchmark and Method

TL;DR

A Concept Drift and Long-Tailed Distribution dataset is presented and a feature recombination framework is proposed to address the learning challenges associated with CDLT and proves the efficacy of the method while also highlighting the limitations of popular large vision-language models in the context of long-tailed distributions.

Abstract

Data is the foundation for the development of computer vision, and the establishment of datasets plays an important role in advancing the techniques of fine-grained visual categorization~(FGVC). In the existing FGVC datasets used in computer vision, it is generally assumed that each collected instance has fixed characteristics and the distribution of different categories is relatively balanced. In contrast, the real world scenario reveals the fact that the characteristics of instances tend to vary with time and exhibit a long-tailed distribution. Hence, the collected datasets may mislead the optimization of the fine-grained classifiers, resulting in unpleasant performance in real applications. Starting from the real-world conditions and to promote the practical progress of fine-grained visual categorization, we present a Concept Drift and Long-Tailed Distribution dataset. Specifically, the dataset is collected by gathering 11195 images of 250 instances in different species for 47 consecutive months in their natural contexts. The collection process involves dozens of crowd workers for photographing and domain experts for labeling. Meanwhile, we propose a feature recombination framework to address the learning challenges associated with CDLT. Experimental results validate the efficacy of our method while also highlighting the limitations of popular large vision-language models (e.g., CLIP) in the context of long-tailed distributions. This emphasizes the significance of CDLT as a benchmark for investigating these challenges.

Figures (8)

• Figure 1: Examples of different concept drift types. Red and blue represent two different concepts, while the light color represents the intermediate concept which appears during the transformation from one to another.
• Figure 2: Examples from the CDLT dataset. The first part shows the large variances in the same subcategory, and the second part illustrates the small variances among different subcategories.
• Figure 3: Illustration of the concept drifts in CDLT.
• Figure 4: The data distribution of common FGVC datasets.
• Figure 5: Performance of the deep models under different partitioning ratios on CDLT.