An unsupervised approach towards promptable defect segmentation in laser-based additive manufacturing by Segment Anything

TL;DR

A framework for image segmentation using a state-of-the-art Vision Transformer based Foundation model with a novel multi-point prompt generation scheme using unsupervised clustering is constructed and porosity segmentation is performed in a case study of laser-based powder bed fusion.

Abstract

Foundation models are currently driving a paradigm shift in computer vision tasks for various fields including biology, astronomy, and robotics among others, leveraging user-generated prompts to enhance their performance. In the Laser Additive Manufacturing (LAM) domain, accurate image-based defect segmentation is imperative to ensure product quality and facilitate real-time process control. However, such tasks are often characterized by multiple challenges including the absence of labels and the requirement for low latency inference among others. Porosity is a very common defect in LAM due to lack of fusion, entrapped gas, and keyholes, directly affecting mechanical properties like tensile strength, stiffness, and hardness, thereby compromising the quality of the final product. To address these issues, we construct a framework for image segmentation using a state-of-the-art Vision Transformer (ViT) based Foundation model (Segment Anything Model) with a novel multi-point prompt generation scheme using unsupervised clustering. Utilizing our framework we perform porosity segmentation in a case study of laser-based powder bed fusion (L-PBF) and obtain high accuracy without using any labeled data to guide the prompt tuning process. By capitalizing on lightweight foundation model inference combined with unsupervised prompt generation, we envision constructing a real-time anomaly detection pipeline that could revolutionize current laser additive manufacturing processes, thereby facilitating the shift towards Industry 4.0 and promoting defect-free production along with operational efficiency.

Figures (9)

• Figure 3.1: XCT images showing the surface topology of defects of two CoCr alloy cylindrical discs; from left to right respectively from kim2017investigation.
• Figure 3.2: The overview of SAM's architecture redrawn from the original paper kirillov2023segment.
• Figure 4.1: Our proposed framework of unsupervised prompt generation with Segment Anything Model.
• Figure 4.2: K-means thresholding and Binarization of the XCT image to create Reference Binary masks; Original XCT image, K-means Thresholding, and Final Binary Mask from left to right respectively.
• Figure 5.1: Prediction results by our framework. Rows 1,2,3,4 represent examples of XCT images from all the samples. (a.1) Input XCT Image from Sample 3, (b.1) Predicted mask by our framework using K-means-based prompts, (c.1) Predicted mask by our framework using K-medoids-based prompts, (d.1) Predicted mask by our framework using corresponding Reference Binary Masks-based prompts,(e.1) Corresponding Reference Binary mask, (a.2) Input XCT Image from Sample 4, (b.2) Predicted mask by our framework using K-means-based prompts, (c.2) Predicted mask by our framework using K-medoids-based prompts, (d.2) Predicted mask by our framework using corresponding Reference Binary Masks-based prompts,(e.2) Corresponding Reference Binary mask, (a.3) Input XCT Image from Sample 5, (b.3) Predicted mask by our framework using K-means-based prompts, (c.3) Predicted mask by our framework using K-medoids-based prompts, (d.3) Predicted mask by our framework using corresponding Reference Binary Masks-based prompts,(e.3) Corresponding Reference Binary mask, (a.4) Input XCT Image from Sample 6, (b.4) Predicted mask by our framework using K-means-based prompts, (c.4) Predicted mask by our framework using K-medoids-based prompts, (d.4) Predicted mask by our framework using corresponding Reference Binary Masks-based prompts,(e.4) Corresponding Reference Binary mask.