Context-Semantic Quality Awareness Network for Fine-Grained Visual Categorization

TL;DR

Context-Semantic Quality Awareness Network (CSQA-Net) tackles fine-grained visual categorization by coupling discriminative local parts with global semantics under weak supervision. It introduces Multi-Level Semantic Quality Evaluation (MLSQE) to supervise semantics across backbone stages, Part Navigator to locate scale-robust discriminative regions, and Multi-Part and Multi-Scale Cross-Attention (MPMSCA) to fuse part descriptors with global object representations. Quality Probing (QP) classifiers provide online feedback to regularize representations during training, while testing relies on the image branch for efficiency, with predictions from multiple main classifiers fused for final output. Across four popular FGVC benchmarks, CSQA-Net delivers consistent gains and often SOTA performance, validating the effectiveness of context-aware, quality-guided learning for fine-grained recognition.

Abstract

Exploring and mining subtle yet distinctive features between sub-categories with similar appearances is crucial for fine-grained visual categorization (FGVC). However, less effort has been devoted to assessing the quality of extracted visual representations. Intuitively, the network may struggle to capture discriminative features from low-quality samples, which leads to a significant decline in FGVC performance. To tackle this challenge, we propose a weakly supervised Context-Semantic Quality Awareness Network (CSQA-Net) for FGVC. In this network, to model the spatial contextual relationship between rich part descriptors and global semantics for capturing more discriminative details within the object, we design a novel multi-part and multi-scale cross-attention (MPMSCA) module. Before feeding to the MPMSCA module, the part navigator is developed to address the scale confusion problems and accurately identify the local distinctive regions. Furthermore, we propose a generic multi-level semantic quality evaluation module (MLSQE) to progressively supervise and enhance hierarchical semantics from different levels of the backbone network. Finally, context-aware features from MPMSCA and semantically enhanced features from MLSQE are fed into the corresponding quality probing classifiers to evaluate their quality in real-time, thus boosting the discriminability of feature representations. Comprehensive experiments on four popular and highly competitive FGVC datasets demonstrate the superiority of the proposed CSQA-Net in comparison with the state-of-the-art methods.

Figures (8)

• Figure 1: A brief view of CSQA-Net. For the image branch, we use multi-level semantic quality evaluation module to enhance hierarchical semantics extracted from the backbone. For the part branch, part navigator is utilized to locate the discriminative regions, and multi-part and multi-scale cross-attention is proposed to generate context-aware features. Finally, we use quality probing classifiers to assess and improve the quality of visual representations. (a) and (b) represent the key response regions generated by ResNet-50 and our proposed CSQA-Net, respectively.
• Figure 2: The detailed architecture of CSQA-Net, which consists of feature extractor (Backbone parameters in light blue and black font are shared), multi-level semantic quality evaluation module, part navigator, and multi-part and multi-scale cross-attention module. $S$ denotes the number of stages included in the backbone network. For clarity, we set $A$ to 3. $\alpha$ represents the confidence for the output of different stages. The detailed structure of the quality probing classifier (QP Classifier) and scale-aware enhancement (SAE) block are shown in Fig. \ref{['fig: quality probing classifier']} and Fig. \ref{['fig:part navigator']}, respectively.
• Figure 3: Illustration of quality probing (QP) classifier. Solid and dotted lines indicate with and without gradient back-propagation, respectively.
• Figure 4: Illustration of dividing the regions according to $\varepsilon^{s'}$. $x$ and $x$+1 represent two adjacent epochs in the training phase.
• Figure 5: Illustration of scale-aware enhancement (SAE) block, which is embedded in part navigator to alleviate the scale confusion problem.