Progressively Exploring and Exploiting Inference Data to Break Fine-Grained Classification Barrier

TL;DR

A novel learning paradigm is proposed that enables the model to progressively learn during inference, thereby leveraging cost-free data at inference time to more accurately represent fine-grained categories and adapt to dynamic semantic changes.

Abstract

Current fine-grained classification research primarily focuses on fine-grained feature learning. However, in real-world scenarios, fine-grained data annotation is challenging, and the features and semantics are highly diverse and frequently changing. These issues create inherent barriers between traditional experimental settings and real-world applications, limiting the effectiveness of conventional fine-grained classification methods. Although some recent studies have provided potential solutions to these issues, most of them still rely on limited supervised information and thus fail to offer effective solutions. In this paper, based on theoretical analysis, we propose a novel learning paradigm to break the barriers in fine-grained classification. This paradigm enables the model to progressively learn during inference, thereby leveraging cost-free data at inference time to more accurately represent fine-grained categories and adapt to dynamic semantic changes. On this basis, an efficient EXPloring and EXPloiting strategy and method (EXP2) is designed. Thereinto, useful inference data samples are explored according to class representations and exploited to optimize classifiers. Experimental results demonstrate the general effectiveness of our method, providing guidance for future in-depth understanding and exploration of real-world fine-grained classification.

Figures (8)

• Figure 1: The diagrams of traditional fine-grained classification, FSFG, and more flexible FSCIL paradigms, along with our proposed learning paradigm of learning during inference.
• Figure 2: Illustration of existing solutions and EXP2 method. (a) Existing solutions use scarce training samples to obtain classifiers, which often causes classifiers to deviate from the real distribution, leading to misclassification. (b)(c) EXP2 dynamically explores useful knowledge from test samples to optimize classifiers, thereby obtaining more accurate classification boundaries and enabling correct classification.
• Figure 3: Illustration of an example to show the impact of estimation error on the probability of inter-class overlap. In this figure, $\delta_{\text{inter}} = 1$, $\sigma_{\text{intra}} = 0.5$ for A, and $\delta_{\text{inter}} = 5$, $\sigma_{\text{intra}} = 0.1$ for B, i.e., A has a smaller $\delta_{\text{inter}}$ and a larger $\sigma_{\text{intra}}$ than B, making it more susceptible to inter-class overlap caused by estimation errors.
• Figure 4: The average of incremental class accuracy across different sessions and the overall accuracy of each session.
• Figure 5: The impact of hyperparameters $R$ and $\tau$ on performance.

Theorems & Definitions (2)

• Lemma 3.1
• proof