Change-Aware Siamese Network for Surface Defects Segmentation under Complex Background

TL;DR

This work reframes surface defect segmentation as a change-detection problem to overcome data scarcity and diverse defect appearances, introducing a change-aware Siamese network that learns defect-free vs. defective feature differences with a Transformer-based encoder guided by a multi-class balanced contrastive loss. A change-aware decoder using DistMap and 3D coordAttention fuses change information with encoded features to robustly locate both intra-class and out-of-class defects. The authors also contribute the SynLCD synthetic LCD defect dataset and demonstrate state-of-the-art performance on SynLCD, PKU-Market-PCB, and MVtec-AD under fully- and semi-supervised settings, with ablations confirming the effectiveness of the balanced contrastive loss and the change-aware decoding strategy. The approach offers practical impact for industrial inspection by enabling accurate pixel-wise defect segmentation with limited labeled data and strong generalization to unseen defect types.

Abstract

Despite the eye-catching breakthroughs achieved by deep visual networks in detecting region-level surface defects, the challenge of high-quality pixel-wise defect detection remains due to diverse defect appearances and data scarcity. To avoid over-reliance on defect appearance and achieve accurate defect segmentation, we proposed a change-aware Siamese network that solves the defect segmentation in a change detection framework. A novel multi-class balanced contrastive loss is introduced to guide the Transformer-based encoder, which enables encoding diverse categories of defects as the unified class-agnostic difference between defect and defect-free images. The difference presented by a distance map is then skip-connected to the change-aware decoder to assist in the location of both inter-class and out-of-class pixel-wise defects. In addition, we proposed a synthetic dataset with multi-class liquid crystal display (LCD) defects under a complex and disjointed background context, to demonstrate the advantages of change-based modeling over appearance-based modeling for defect segmentation. In our proposed dataset and two public datasets, our model achieves superior performances than the leading semantic segmentation methods, while maintaining a relatively small model size. Moreover, our model achieves a new state-of-the-art performance compared to the semi-supervised approaches in various supervision settings.

Figures (12)

• Figure 1: The examples illustrate how our change-based and appearance-based methods have segmented defects in fully-supervised, semi-supervised, and unsupervised settings. The results in column (d) are derived from SegFormer SegFormer. The outcomes in column (e) originate from UAPS sime2023uncertainty. In the prediction maps, green signifies missed detections and red indicates erroneous detections. The term "Err" quantifies the total of these errors. Our model outperforms semi-supervised methods and achieves competitive outcomes using only 10% of the training samples compared to the fully-supervised model.
• Figure 2: The pipeline of our change-aware Siamese network, which consists of a difference-indicating encoder that extracts contrastive features and a change-aware decoder that applies feature difference (DistMap) to assist in defect localization. The cross-entropy and balanced contrastive loss are adapted for training.
• Figure 3: The basic modules. (a) The sequence reduction attention utilizes the spatial reduction layer to reduce the complexity of the self-attention module from $O(N^2)$ to $O\left(\frac{N^2}{R}\right)$. (b) The change-aware decoder, based on a 3-dimensional (horizontal, vertical, and depth) attention module, utilizes the distMap carrying change information in different ways when detecting intra-class and OOC objects.
• Figure 4: Ten defect-free LCD display patterns. In real inspection process, the industrial LCD display patterns are constructed with RGB blocks, gray transition, color maps, characters, and faces to reveal various types of defects (e.g, point, line, and Mura defects ming2021survey).
• Figure 5: Samples of SynLCD and the dataset challenges. (a) Abnormal points defect sample; (b) line defect sample; (c) mixed defect sample; (d) binary label of mixed defect image. (e) RGB deviation and irregular screen texture; (f) nonlinear saturation difference. (g) low contrast abpt and line defects.