Fine-grained Classes and How to Find Them

TL;DR

FALCON is a method that discovers fine-grained classes from coarsely labeled data without any supervision at the fine-grained level, and simultaneously infers unknown fine-grained classes and underlying relationships between coarse and fine-grained classes.

Abstract

In many practical applications, coarse-grained labels are readily available compared to fine-grained labels that reflect subtle differences between classes. However, existing methods cannot leverage coarse labels to infer fine-grained labels in an unsupervised manner. To bridge this gap, we propose FALCON, a method that discovers fine-grained classes from coarsely labeled data without any supervision at the fine-grained level. FALCON simultaneously infers unknown fine-grained classes and underlying relationships between coarse and fine-grained classes. Moreover, FALCON is a modular method that can effectively learn from multiple datasets labeled with different strategies. We evaluate FALCON on eight image classification tasks and a single-cell classification task. FALCON outperforms baselines by a large margin, achieving 22% improvement over the best baseline on the tieredImageNet dataset with over 600 fine-grained classes.

Figures (7)

• Figure 1: FALCON simultaneously discovers fine-grained classes and infers relationships between the discovered fine and the available coarse classes by coarse supervision. The fine-grained classifier optimizes the loss (\ref{['eq:final_cls']}), while the class relationships are inferred by solving a discrete optimization problem (\ref{['eq:objective_M']}).
• Figure 2: The t-SNE plot of Living17 test samples in the embedding space learned by FALCON. Coarse-grained classes used to supervise the model are shown in different colors. The images at the left and right side show representative examples of inferred fine-grained classes for coarse classes Spider and Grouse.
• Figure 3: Three most confident predictions for fine-grained classes associated with coarse class Salamander and Bear.
• Figure 4: Subclasses discovered within fine-grained classes Eft and Ptarmigan. The discovered subclasses differ according to the skin (Eft) or feather (Ptarmigan) color.
• Figure F1: Number of images in every fine-grained class of sample imbalanced version of the CIFAR100 dataset.