Bidirectional Logits Tree: Pursuing Granularity Reconcilement in Fine-Grained Classification

TL;DR

A novel framework called the Bidirectional Logits Tree (BiLT) for Granularity Reconcilement is proposed, which develops classifiers sequentially from the finest to the coarsest granularities, rather than parallelly constructing a set of classifiers based on the same input features.

Abstract

This paper addresses the challenge of Granularity Competition in fine-grained classification tasks, which arises due to the semantic gap between multi-granularity labels. Existing approaches typically develop independent hierarchy-aware models based on shared features extracted from a common base encoder. However, because coarse-grained levels are inherently easier to learn than finer ones, the base encoder tends to prioritize coarse feature abstractions, which impedes the learning of fine-grained features. To overcome this challenge, we propose a novel framework called the Bidirectional Logits Tree (BiLT) for Granularity Reconcilement. The key idea is to develop classifiers sequentially from the finest to the coarsest granularities, rather than parallelly constructing a set of classifiers based on the same input features. In this setup, the outputs of finer-grained classifiers serve as inputs for coarser-grained ones, facilitating the flow of hierarchical semantic information across different granularities. On top of this, we further introduce an Adaptive Intra-Granularity Difference Learning (AIGDL) approach to uncover subtle semantic differences between classes within the same granularity. Extensive experiments demonstrate the effectiveness of our proposed method.

Figures (10)

• Figure 1: The overall framework of our method. In the forward phase of the Bidirectional Logits Tree (BiLT), coarse-grained logits are derived from fine-grained counterparts, while the gradients from coarse-grained classifiers influence fine-grained classifiers and feature learning in the backward phase. Simultaneously, Adaptive Intra-Granularity Difference Learning (AIGDL) adjusts the output of BiLT and supervision by learning differences between categories within the same granularity.
• Figure 2: Comparison of convergence speed of Sharing Same Features and BiLT
• Figure 3: An illustrative example demonstrates the granularity competition problem, where the model prioritizes coarse-grained learning at Level 1 and Level 2, thereby rendering the fine-grained features at Level 3 difficult to distinguish.
• Figure 4: Predefined label trees struggle to articulate differences amongst classes at the same hierarchical level, and our method aims to learn disparities among classes and apply relevant corrections accordingly.
• Figure 5: The probability of mistakes made by our method and competitors across the three datasets. In each subfigure, different colors represent varying LCA distances, increasing from left to right. The length of each bar indicates the severity of the mistake, while the values on the bars show the probability of the model making a mistake at a specific LCA distance.