PartImageNet++ Dataset: Enhancing Visual Models with High-Quality Part Annotations

TL;DR

PIN++ provides 100K high-quality part-annotated images across 1K ImageNet-1K categories, enabling large-scale part-based learning. The authors train a part segmentation model on PIN++ to generate pseudo labels for the remaining data and introduce the Multi-scale Part-supervised recognition Model (MPM), which injects part supervision into intermediate features via lightweight bypass layers with the objective $L = L_{cls} + \lambda L_{seg}$. Empirical results show that MPM improves adversarial robustness, corruption tolerance, and alignment with human vision on IN-1K, while PIN++ enables strong part- and object-segmentation benchmarks and improves few-shot learning when part cues are incorporated. The work demonstrates the practical impact of rich part annotations for robust recognition and dense prediction, providing datasets and code to catalyze further research in part-based visual understanding.

Abstract

To address the scarcity of high-quality part annotations in existing datasets, we introduce PartImageNet++ (PIN++), a dataset that provides detailed part annotations for all categories in ImageNet-1K. With 100 annotated images per category, totaling 100K images, PIN++ represents the most comprehensive dataset covering a diverse range of object categories. Leveraging PIN++, we propose a Multi-scale Part-supervised recognition Model (MPM) for robust classification on ImageNet-1K. We first trained a part segmentation network using PIN++ and used it to generate pseudo part labels for the remaining unannotated images. MPM then integrated a conventional recognition architecture with auxiliary bypass layers, jointly supervised by both pseudo part labels and the original part annotations. Furthermore, we conducted extensive experiments on PIN++, including part segmentation, object segmentation, and few-shot learning, exploring various ways to leverage part annotations in downstream tasks. Experimental results demonstrated that our approach not only enhanced part-based models for robust object recognition but also established strong baselines for multiple downstream tasks, highlighting the potential of part annotations in improving model performance. The dataset and the code are available at https://github.com/LixiaoTHU/PartImageNetPP.

Figures (7)

• Figure 1: Examples of annotated images in PIN++. The names of the objects are displayed in the upper-right corner of each image, while the part names are not revealed in this context.
• Figure 2: Visual comparison of annotations between PIN and PIN++. The object names are displayed on the top-right of the IN-1K columns. The part names are presented in the images of PIN and PIN++.
• Figure 3: Comparison between part segmentation results of different methods and PIN++ annotations. Without training on PIN++, VLPart and SAM fail to segment objects into specific parts with accurate semantics.
• Figure 4: An overview of the generation of pseudo-labels and the structure of MPM. (a) A part segmentation model trained on PIN++ and used to obtain pseudo-part labels for unannotated images. (b) MPM adds several auxiliary bypass layers to the vanilla recognition model for part segmentation supervision. MPM is trained by part annotations together with the pseudo part labels. During inference, the auxiliary layers are dropped, and the vanilla recognition model gives the final object category prediction.
• Figure 5: Visualization of pseudo part labels generated by a Mask R-CNN trained on PIN++. The object names are shown on the top-right of each image. The part names are hidden here for clarity.