Look Inside for More: Internal Spatial Modality Perception for 3D Anomaly Detection

TL;DR

The paper tackles the limited use of internal spatial information in 3D anomaly detection for point clouds. It proposes Internal Spatial Modality Perception (ISMP), comprising the Spatial Insight Engine (SIE), an enhanced feature extractor, and a feature filtering module to capture and align internal-global spatial cues while suppressing noise. ISMP achieves state-of-the-art object- and pixel-level AUROC on Real3D-AD and competitive results on Anomaly-ShapeNet, with clear evidence of SIE’s generalization in ModelNet40 and ShapeNet-Part. The framework holds promise for more robust anomaly localization in industrial and medical contexts, though future work is needed to accelerate inference speed.

Abstract

3D anomaly detection has recently become a significant focus in computer vision. Several advanced methods have achieved satisfying anomaly detection performance. However, they typically concentrate on the external structure of 3D samples and struggle to leverage the internal information embedded within samples. Inspired by the basic intuition of why not look inside for more, we introduce a straightforward method named Internal Spatial Modality Perception~(ISMP) to explore the feature representation from internal views fully. Specifically, our proposed ISMP consists of a critical perception module, Spatial Insight Engine~(SIE), which abstracts complex internal information of point clouds into essential global features. Besides, to better align structural information with point data, we propose an enhanced key point feature extraction module for amplifying spatial structure feature representation. Simultaneously, a novel feature filtering module is incorporated to reduce noise and redundant features for further aligning precise spatial structure. Extensive experiments validate the effectiveness of our proposed method, achieving object-level and pixel-level AUROC improvements of 3.2\% and 13.1\%, respectively, on the Real3D-AD benchmarks. Note that the strong generalization ability of SIE has been theoretically proven and is verified in both classification and segmentation tasks.

Figures (4)

• Figure 1: Visualization of internal and external perception. Compared with external view (CPMF), our method (ISMP) projects from the internal view, better capturing the different-shaped protrusions in the 3D structure.
• Figure 2: Overview of our method. We start by matching the point cloud according to RANSAC 9318535. During the training phase, we create feature and coordinate memory banks, using enhanced feature extraction to capture local information and constructing a local feature matrix. We extract global features using the SIE and align them with the local ones. $P_{1}$, $P_{2}$, $P_{3}$, and $P_{4}$ are the four projection slices extracted respectively. Then, we employ a feature filtering module to suppress redundant information, resulting in the final feature matrix. In the inference phase, we obtain the final feature matrix and compute the nearest neighbors in the memory bank. Finally, we input the coordinates into the coordinate memory bank to find the closest regular sample coordinates, calculating the final score of the sample points based on both memory banks.
• Figure 3: Visualization of projection slices. The images are the original image, $P_1$, $P_2$, $P_3$, and $P_4$, respectively.
• Figure 4: Heatmaps of the impact of parameters $\alpha$, $\beta$, $\gamma$ on mean and variance. The ordinate represents a combination of $\alpha$ and $\beta$, and the abscissa represents $\gamma$. The lighter the color of the block in the figure, the larger the difference before and after the transformation. The red marks in the figure are the parameters we selected.