Fine-Grained Generalization via Structuralizing Concept and Feature Space into Commonality, Specificity and Confounding

TL;DR

The paper tackles fine-grained domain generalization (FGDG) by proposing Concept-Feature Structuralized Generalization (CFSG), which disentangles both concept and feature spaces into commonality, specificity, and confounding components. An adaptive mechanism allocates weights to these components during classification, and a Granularity Transition Layer enables multi-granularity learning. The approach is reinforced by neural-collapse–inspired ideas and a concept bottleneck perspective, yielding large improvements over baselines and strong explainability of learned structure. Empirical results on three single-source FGDG benchmarks demonstrate robust generalization across diverse backbones, with substantial performance gains and interpretable multi-granularity representations.

Abstract

Fine-Grained Domain Generalization (FGDG) presents greater challenges than conventional domain generalization due to the subtle inter-class differences and relatively pronounced intra-class variations inherent in fine-grained recognition tasks. Under domain shifts, the model becomes overly sensitive to fine-grained cues, leading to the suppression of critical features and a significant drop in performance. Cognitive studies suggest that humans classify objects by leveraging both common and specific attributes, enabling accurate differentiation between fine-grained categories. However, current deep learning models have yet to incorporate this mechanism effectively. Inspired by this mechanism, we propose Concept-Feature Structuralized Generalization (CFSG). This model explicitly disentangles both the concept and feature spaces into three structured components: common, specific, and confounding segments. To mitigate the adverse effects of varying degrees of distribution shift, we introduce an adaptive mechanism that dynamically adjusts the proportions of common, specific, and confounding components. In the final prediction, explicit weights are assigned to each pair of components. Extensive experiments on three single-source benchmark datasets demonstrate that CFSG achieves an average performance improvement of 9.87% over baseline models and outperforms existing state-of-the-art methods by an average of 3.08%. Additionally, explainability analysis validates that CFSG effectively integrates multi-granularity structured knowledge and confirms that feature structuralization facilitates the emergence of concept structuralization.

Figures (5)

• Figure 1: Decision mechanisms of different methods: conventional models predict directly from extracted features; FSDG structures the feature space, but classification still relies on features. In contrast, CFSG first achieves concept structuralization through structured features and then performs classification based on these structured concepts. FE stands for Feature Extractor, FS refers to Feature Structuralization, and CS represents Concept Structuralization.
• Figure 2: The model illustration presents an example based on a three-level granularity hierarchy. Granu. Trans. denotes the Granularity Transition Layer. Given an input image, CFSG performs structuralization and disentanglement in both the feature and concept spaces, and conducts classification by assigning different weights to the structured representations.
• Figure 3: Classification accuracy (%) of CFSG under varying commonality, specificity, and confounding weights when trained on P and tested on C.
• Figure 4: Cosine similarity results of FGDG and CFSG across common, specific, and confounding concepts.
• Figure 5: Confusion matrix of the ground truth for concept similarity.