Multi-Sensor Object Anomaly Detection: Unifying Appearance, Geometry, and Internal Properties

TL;DR

This work introduces MulSen-AD, the first industrial multi-sensor anomaly-detection dataset that jointly captures appearance (RGB), internal/subsurface information (lock-in IR), and geometry (high-resolution point clouds). It provides MulSen-TripleAD, a late-decision fusion baseline that combines RGB, IR, and PC features via modality-specific memory banks and a gating unit, achieving an average object-level AUROC of 0.961 and demonstrating the superiority of multi-sensor fusion over single-modality approaches. The paper also offers MulSen-AD Bench for standardized evaluation and shows strong localization performance with RGB dominating pixel-level detection, complemented by infrared cues, while 3D data enhances geometric anomaly detection. Overall, the dataset and baseline establish a new direction for robust, real-world industrial anomaly detection and localization, with practical implications for manufacturing QA and automated inspection systems.

Abstract

Object anomaly detection is essential for industrial quality inspection, yet traditional single-sensor methods face critical limitations. They fail to capture the wide range of anomaly types, as single sensors are often constrained to either external appearance, geometric structure, or internal properties. To overcome these challenges, we introduce MulSen-AD, the first high-resolution, multi-sensor anomaly detection dataset tailored for industrial applications. MulSen-AD unifies data from RGB cameras, laser scanners, and lock-in infrared thermography, effectively capturing external appearance, geometric deformations, and internal defects. The dataset spans 15 industrial products with diverse, real-world anomalies. We also present MulSen-AD Bench, a benchmark designed to evaluate multi-sensor methods, and propose MulSen-TripleAD, a decision-level fusion algorithm that integrates these three modalities for robust, unsupervised object anomaly detection. Our experiments demonstrate that multi-sensor fusion substantially outperforms single-sensor approaches, achieving 96.1% AUROC in object-level detection accuracy. These results highlight the importance of integrating multi-sensor data for comprehensive industrial anomaly detection.

Figures (16)

• Figure 1: Motivation for multi-sensor object anomaly detection. Different sensors capture distinct anomalies, making fusion essential. Our MulSen-AD dataset demonstrates how RGB captures surface defects, point clouds detect geometric deformations, and infrared reveals internal and subsurface issues. Red boxes enclose anomalies, blue highlights normal references.
• Figure 2: Data collection pipeline for the proposed MulSen-AD dataset consists of three stages: (a) Infrared image acquisition, (b) RGB image capture, and (c) Point cloud collection and alignment. The pink 'Piggy' object serves as the example for data collection.
• Figure 3: 15 object categories from MulSen-AD, each represented in three modalities—RGB, IR, and point cloud. Some defects are visible in only one or two modalities. We highlight the abnormal areas using red overlay masks.
• Figure 4: Anomaly distribution captured by single and multiple sensors in MulSen-AD. Overlap regions represent anomalies that are observable by multiple sensors.
• Figure 5: Anomaly data distribution of MulSen-AD dataset. The annotation count for each modality reflects the number of detectable anomaly samples per modality. (a) Anomaly annotation counts by modality across categories. (b) Distribution of anomaly types per category.