Boosting Global-Local Feature Matching via Anomaly Synthesis for Multi-Class Point Cloud Anomaly Detection

TL;DR

The paper tackles multi-class point cloud anomaly detection, where a single model must discriminate anomalies across many classes despite inter-class feature similarity. It introduces GLFM, a three-stage framework that first synthesizes realistic 3D anomalies to adapt a backbone, then builds global and local memory banks to separate classes, and finally detects anomalies by measuring distances to these banks. The approach yields state-of-the-art results on both MVTec 3D-AD and Real3D-AD, with strong generalization to real industrial parts and few-shot scenarios. The work advances practical industrial AD by addressing feature confusion and demonstrates the importance of combining global context with local detail for robust multi-class detection.

Abstract

Point cloud anomaly detection is essential for various industrial applications. The huge computation and storage costs caused by the increasing product classes limit the application of single-class unsupervised methods, necessitating the development of multi-class unsupervised methods. However, the feature similarity between normal and anomalous points from different class data leads to the feature confusion problem, which greatly hinders the performance of multi-class methods. Therefore, we introduce a multi-class point cloud anomaly detection method, named GLFM, leveraging global-local feature matching to progressively separate data that are prone to confusion across multiple classes. Specifically, GLFM is structured into three stages: Stage-I proposes an anomaly synthesis pipeline that stretches point clouds to create abundant anomaly data that are utilized to adapt the point cloud feature extractor for better feature representation. Stage-II establishes the global and local memory banks according to the global and local feature distributions of all the training data, weakening the impact of feature confusion on the establishment of the memory bank. Stage-III implements anomaly detection of test data leveraging its feature distance from global and local memory banks. Extensive experiments on the MVTec 3D-AD, Real3D-AD and actual industry parts dataset showcase our proposed GLFM's superior point cloud anomaly detection performance. The code is available at https://github.com/hustCYQ/GLFM-Multi-class-3DAD.

Figures (12)

• Figure 1: a) Illustration of the single-class AD task, in which each class needs a specialized model. b) Illustration of the multi-class AD task, in which a unified model is trained and tested in the data from multiple classes. c) Vanilla multi-class AD method struggles to distinguish normal and anomaly data due to feature confusion. d) The proposed method clusters by global features and performs AD on each class to solve feature confusion.
• Figure 2: Feature visualization of anomaly data from Dowel class and normal data from Carrot class. Features are extracted through PointMAE pointmae that is pre-trained on ShapeNet Shapenet and principal components analysis (PCA) is used to reduce the feature dimension to two for visualization. Yellow shapes represent the features of Dowel data, and green shapes represent the features of Carrot. Anomalous regions/features are highlighted within a red box.
• Figure 3: The framework of the proposed GLFM. a) Stage-I: Anomaly data and corresponding point-wise labels are synthesized by select part points of the point clouds for irregular stretching to produce local protrusions or depressions. Then point cloud feature extractor (point transformer) is adapted by self-supervised learning of point cloud anomaly segmentation. b) Stage-II: During the training process, the global and local features of all training data are extracted by point cloud feature extractor adapted. Global features are clustered to construct $\boldsymbol{\mathcal{M}}_{G}$. Local features of training data are divided into multiple local memory banks $\boldsymbol{\mathcal{M}}_{i}$ based on the distance between the cluster centers and their global features. c) Stage-III: During the testing process, both global and local features of input point cloud are extracted seems to training process, and $\boldsymbol{\mathcal{M}}_{G}$ is used to answer the query of global features. Local features are then used to perform anomaly detection in the corresponding $\boldsymbol{\mathcal{M}}_{i}$.
• Figure 4: Visualization of synthetic anomaly data from different classes. The red points represent the anomalies. The synthetic anomaly data have the properties of local continuity and smoothness.
• Figure 5: Visualization of prediction results in MVTec 3D-AD dataset using the proposed method and other methods. The first row is the original point clouds, while the second row is the ground truth. Subsequent rows depict various methods.