Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset

TL;DR

This work tackles open-vocabulary industrial defect understanding by introducing IMDD-1M, the first large-scale industrial dataset with aligned image-text pairs, spanning 63 domains and 421 defect types. It pairs IMDD-1M with a diffusion-based vision-language foundation model that integrates discriminative tasks (segmentation, detection) and generative capabilities (synthesis, augmentation) in a single architecture. A hybrid annotation pipeline and an implicit captioner enable domain-specific, language-grounded learning with data efficiency, achieving competitive zero-shot and few-shot performance across defect classification, localization, and segmentation using less than 5% of task-specific data. The approach demonstrates strong cross-domain generalization, robust generation quality, and practical potential for scalable, knowledge-grounded manufacturing intelligence, though it remains computationally intensive and focused on 2D RGB data. Future directions include temporal and 3D extensions, cross-domain adaptation, and broader modality integration to further industrial inspection capabilities.

Abstract

We present IMDD-1M, the first large-scale Industrial Multimodal Defect Dataset comprising 1,000,000 aligned image-text pairs, designed to advance multimodal learning for manufacturing and quality inspection. IMDD-1M contains high-resolution real-world defects spanning over 60 material categories and more than 400 defect types, each accompanied by expert-verified annotations and fine-grained textual descriptions detailing defect location, severity, and contextual attributes. This dataset enables a wide spectrum of applications, including classification, segmentation, retrieval, captioning, and generative modeling. Building upon IMDD-1M, we train a diffusion-based vision-language foundation model from scratch, specifically tailored for industrial scenarios. The model serves as a generalizable foundation that can be efficiently adapted to specialized domains through lightweight fine-tuning. With less than 5% of the task-specific data required by dedicated expert models, it achieves comparable performance, highlighting the potential of data-efficient foundation model adaptation for industrial inspection and generation, paving the way for scalable, domain-adaptive, and knowledge-grounded manufacturing intelligence.

Figures (14)

• Figure 1: Overview of the proposed IMDD-1M dataset, its diverse industrial domains, corresponding downstream tasks, and potential extensions to vision-language model applications.
• Figure 2: Illustrative overview of IMDD-1M showing diverse image-text pairs across multiple industrial domains, each with expert-verified annotations capturing fine-grained defect types, materials, and manufacturing contexts. The dataset serves as a large-scale foundation for vision-language modeling in industrial inspection.
• Figure 3: Dataset analysis. (1) Sample distribution among the top 100 defect categories (log-scaled). (2) Three-step workflow for dataset construction.
• Figure 4: Dataset composition. (1) Distribution of normal versus anomaly samples. (2) Pie chart showing dataset composition across domains.
• Figure 5: Anomaly ratio distribution across datasets. Each bar represents the normalized proportion of anomaly and normal samples within a specific dataset, illustrating data imbalance and diversity across industrial domains.