Not just Birds and Cars: Generic, Scalable and Explainable Models for Professional Visual Recognition

TL;DR

A biologically-inspired structure named Pro-NeXt is introduced and it is revealed that Pro-NeXt exhibits substantial generalizability across diverse professional fields such as fashion, medicine, and art-areas previously considered disparate.

Abstract

Some visual recognition tasks are more challenging then the general ones as they require professional categories of images. The previous efforts, like fine-grained vision classification, primarily introduced models tailored to specific tasks, like identifying bird species or car brands with limited scalability and generalizability. This paper aims to design a scalable and explainable model to solve Professional Visual Recognition tasks from a generic standpoint. We introduce a biologically-inspired structure named Pro-NeXt and reveal that Pro-NeXt exhibits substantial generalizability across diverse professional fields such as fashion, medicine, and art-areas previously considered disparate. Our basic-sized Pro-NeXt-B surpasses all preceding task-specific models across 12 distinct datasets within 5 diverse domains. Furthermore, we find its good scaling property that scaling up Pro-NeXt in depth and width with increasing GFlops can consistently enhances its accuracy. Beyond scalability and adaptability, the intermediate features of Pro-NeXt achieve reliable object detection and segmentation performance without extra training, highlighting its solid explainability. We will release the code to foster further research in this area.

Figures (8)

• Figure 1: Generic Professional Recognition is more challenging than general and FGVC as it faces both Label Complexity (professional labels) and Task Complexity (a wide range of tasks).
• Figure 2: An illustration of our Pro-NeXt framework, which starts from (a) an overview pipeline of Pro-NeXt, and continues with zoomed-in diagrams of (b) Gaze-Shift.
• Figure 3: A comparison of different feature attentive mechanisms. The proposed Shift-Parser (c) takes advantages of both spatial domain methods (a) and frequency domain method (b), which groups the features in frequency domain while also maintaining the spatial constraint.
• Figure 4: The visualized results of Shift-Parser mask in each stage (mapping back to the raw image). The focal parts are kept and the context parts are masked. For the small focal parts, we zoom in the region on the up-left/right corners for clarity. From top to down are wildlife/vehicles, medical image, and artwork classification respectively.
• Figure 4: Accuracy on ImageNet Benchmarks comparing with general image classification methods.