GCAM: Gaussian and causal-attention model of food fine-grained recognition

TL;DR

A Gaussian and causal-attention model is proposed for fine-grained object recognition that surpasses state-of-the-art methods on the ETH-FOOD101, UECFOOD256, and Vireo-FOOD172 datasets and achieves state-of-the-art performance on the CUB-200 dataset.

Abstract

Currently, most food recognition relies on deep learning for category classification. However, these approaches struggle to effectively distinguish between visually similar food samples, highlighting the pressing need to address fine-grained issues in food recognition. To mitigate these challenges, we propose the adoption of a Gaussian and causal-attention model for fine-grained object recognition.In particular, we train to obtain Gaussian features over target regions, followed by the extraction of fine-grained features from the objects, thereby enhancing the feature mapping capabilities of the target regions. To counteract data drift resulting from uneven data distributions, we employ a counterfactual reasoning approach. By using counterfactual interventions, we analyze the impact of the learned image attention mechanism on network predictions, enabling the network to acquire more useful attention weights for fine-grained image recognition. Finally, we design a learnable loss strategy to balance training stability across various modules, ultimately improving the accuracy of the final target recognition. We validate our approach on four relevant datasets, demonstrating its excellent performance across these four datasets.We experimentally show that GCAM surpasses state-of-the-art methods on the ETH-FOOD101, UECFOOD256, and Vireo-FOOD172 datasets. Furthermore, our approach also achieves state-of-the-art performance on the CUB-200 dataset.

Figures (11)

• Figure 1: food system
• Figure 2: Distribution of food categories by location
• Figure 3: Overall architecture of GCAM for food identification. Input the original image, obtain the coarse-grained features through FGF, and then obtain the refined image through clipping. Among them, CRA improves the quality of the network attention mechanism and reduces the global discriminative ability of the network attention mechanism. The final losses are pooled into LLS for optimization.
• Figure 4: Gaussian feature fusion process.
• Figure 5: In the image clipping process, more useful information is obtained through the interaction of feature maps and attention maps, which is used to obtain the next refinement map.