Uni-3DAD: GAN-Inversion Aided Universal 3D Anomaly Detection on Model-free Products

Abstract

Anomaly detection is a long-standing challenge in manufacturing systems. Traditionally, anomaly detection has relied on human inspectors. However, 3D point clouds have gained attention due to their robustness to environmental factors and their ability to represent geometric data. Existing 3D anomaly detection methods generally fall into two categories. One compares scanned 3D point clouds with design files, assuming these files are always available. However, such assumptions are often violated in many real-world applications where model-free products exist, such as fresh produce (i.e., ``Cookie", ``Potato", etc.), dentures, bone, etc. The other category compares patches of scanned 3D point clouds with a library of normal patches named memory bank. However, those methods usually fail to detect incomplete shapes, which is a fairly common defect type (i.e., missing pieces of different products). The main challenge is that missing areas in 3D point clouds represent the absence of scanned points. This makes it infeasible to compare the missing region with existing point cloud patches in the memory bank. To address these two challenges, we proposed a unified, unsupervised 3D anomaly detection framework capable of identifying all types of defects on model-free products. Our method integrates two detection modules: a feature-based detection module and a reconstruction-based detection module. Feature-based detection covers geometric defects, such as dents, holes, and cracks, while the reconstruction-based method detects missing regions. Additionally, we employ a One-class Support Vector Machine (OCSVM) to fuse the detection results from both modules. The results demonstrate that (1) our proposed method outperforms the state-of-the-art methods in identifying incomplete shapes and (2) it still maintains comparable performance with the SOTA methods in detecting all other types of anomalies.

Figures (8)

• Figure 1: Comparison of visual results between our proposed and current SOTA (BTF horwitz2022empirical, M3DM wang2023multimodal, Shape-guided shapeguide) methods on the MVTec 3D-AD dataset Bergmann_Jin_Sattlegger_Steger_2022b. Bright areas indicate high anomaly scores. Our method successfully localizes missing regions as defects without compromising the detection accuracy of the other geometric defects, such as dents, holes, and scratches.
• Figure 2: Inference of our proposed method. Our proposed method has three basic modules: (i) feature-based module, (ii) reconstruction-based module, and (iii) OCSVM fusion module. The design of the proposed method can facilitate the detection of universal defects.
• Figure 3: The visual demonstration of SPFH and FPFH: each dashed circle represents the receptive field of SPFH of the center point, i.e., $a_{1}, a_2, a_3, a_4$, and the solid circle shows the receptive field of FPFH of $a_{1}$.
• Figure 4: Scheme of calculating the anomaly score $\textbf{S}_{1}$.
• Figure 5: The architecture of GAN-Inversion anomaly detection. There are two parts in our proposed module: (i) GAN-Inversion aims to find the latent code $\mathbf{z}^{*}$, and the network parameters $\theta^{*}$ can best reconstruct the testing samples; (ii) Anomaly Detection module involves an analytical comparison between the testing incomplete samples and the reconstructed samples, where the reconstructed samples are generated by the latent code $\textbf{z}^{*}$ and $\theta^{*}$ obtained GAN-Inversion module.