2D_3D Feature Fusion via Cross-Modal Latent Synthesis and Attention Guided Restoration for Industrial Anomaly Detection

TL;DR

This work tackles industrial anomaly detection by leveraging synchronized 2D and 3D data through a unified cross-modal fusion approach. The authors introduce MAFR, an unsupervised framework that encodes fused multimodal features into a shared latent space and uses CBAM-guided decoders to reconstruct modality-specific features, enabling precise anomaly localization via reconstruction errors. Key contributions include a robust composite loss (ZNSSD-based similarity, census, and smoothness losses), a CBAM-enhanced decoupled restoration design, and a multiplicative fusion of per-modality anomaly maps, achieving state-of-the-art results on MVTec 3D-AD and Eyecandies and demonstrating strong few-shot robustness. The method advances industrial inspection by providing accurate, efficient multimodal anomaly detection and localization without heavy memory banks, with practical impact in real-time quality control and deployment under data-scarce conditions.

Abstract

Industrial anomaly detection (IAD) increasingly benefits from integrating 2D and 3D data, but robust cross-modal fusion remains challenging. We propose a novel unsupervised framework, Multi-Modal Attention-Driven Fusion Restoration (MAFR), which synthesises a unified latent space from RGB images and point clouds using a shared fusion encoder, followed by attention-guided, modality-specific decoders. Anomalies are localised by measuring reconstruction errors between input features and their restored counterparts. Evaluations on the MVTec 3D-AD and Eyecandies benchmarks demonstrate that MAFR achieves state-of-the-art results, with a mean I-AUROC of 0.972 and 0.901, respectively. The framework also exhibits strong performance in few-shot learning settings, and ablation studies confirm the critical roles of the fusion architecture and composite loss. MAFR offers a principled approach for fusing visual and geometric information, advancing the robustness and accuracy of industrial anomaly detection. Code is available at https://github.com/adabrh/MAFR

Figures (2)

• Figure 1: An overview of the proposed architecture. RGB and point cloud inputs are processed by feature extractors ($\mathcal{F}_{2D}$, $\mathcal{F}_{3D}$) to produce deep feature maps. These features follow two paths: (1) A fusion path where an encoder, a fusion block ($F_{\text{fus}}$), and two decoupled decoders generate reconstructed features. (2) A reconstruction path where a combined loss is calculated between the original and reconstructed features. The decoders' outputs, refined by CBAM, produce modality-specific anomaly maps ($\Psi_{2D}$, $\Psi_{3D}$), which are fused via element-wise multiplication (A) to create the final combined anomaly map ($\Psi_{\text{COMB}}$).
• Figure 2: Qualitative results comparison of MAFR with the baseline method across 10 categories on MVTec 3D-AD dataset.