Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

In-Situ Melt Pool Characterization via Thermal Imaging for Defect Detection in Directed Energy Deposition Using Vision Transformers

Israt Zarin Era, Fan Zhou, Ahmed Shoyeb Raihan, Imtiaz Ahmed, Alan Abul-Haj, James Craig, Srinjoy Das, Zhichao Liu

TL;DR

Directed Energy Deposition melt-pool porosity detection is challenged by scarce labeled data. The authors propose a self-supervised learning pipeline using a Vision Transformer-based Masked Autoencoder (MAE) pretrained on unlabeled melt-pool images, then transfer-learned to two classifiers trained on a small labeled set. On in-situ thermal images from a custom DED setup, the approach achieves high accuracy (0.954–0.992) with an average accuracy of 98.2% and average F1 of 82.8% for the ViT-based classifier, demonstrating strong defect-detection performance with limited labels. The work highlights the practicality of SSL ViTs for scalable, low-labeled data quality control in DED melt-pool monitoring and points to future improvements with alternative losses and semi-supervised strategies.

Abstract

Directed Energy Deposition (DED) offers significant potential for manufacturing complex and multi-material parts. However, internal defects such as porosity and cracks can compromise mechanical properties and overall performance. This study focuses on in-situ monitoring and characterization of melt pools associated with porosity, aiming to improve defect detection and quality control in DED-printed parts. Traditional machine learning approaches for defect identification rely on extensive labeled datasets, often scarce and expensive to generate in real-world manufacturing. To address this, our framework employs self-supervised learning on unlabeled melt pool data using a Vision Transformer-based Masked Autoencoder (MAE) to produce highly representative embeddings. These fine-tuned embeddings are leveraged via transfer learning to train classifiers on a limited labeled dataset, enabling the effective identification of melt pool anomalies. We evaluate two classifiers: (1) a Vision Transformer (ViT) classifier utilizing the fine-tuned MAE Encoder's parameters and (2) the fine-tuned MAE Encoder combined with an MLP classifier head. Our framework achieves overall accuracy ranging from 95.44% to 99.17% and an average F1 score exceeding 80%, with the ViT Classifier slightly outperforming the MAE Encoder Classifier. This demonstrates the scalability and cost-effectiveness of our approach for automated quality control in DED, effectively detecting defects with minimal labeled data.

In-Situ Melt Pool Characterization via Thermal Imaging for Defect Detection in Directed Energy Deposition Using Vision Transformers

TL;DR

Directed Energy Deposition melt-pool porosity detection is challenged by scarce labeled data. The authors propose a self-supervised learning pipeline using a Vision Transformer-based Masked Autoencoder (MAE) pretrained on unlabeled melt-pool images, then transfer-learned to two classifiers trained on a small labeled set. On in-situ thermal images from a custom DED setup, the approach achieves high accuracy (0.954–0.992) with an average accuracy of 98.2% and average F1 of 82.8% for the ViT-based classifier, demonstrating strong defect-detection performance with limited labels. The work highlights the practicality of SSL ViTs for scalable, low-labeled data quality control in DED melt-pool monitoring and points to future improvements with alternative losses and semi-supervised strategies.

Abstract

Directed Energy Deposition (DED) offers significant potential for manufacturing complex and multi-material parts. However, internal defects such as porosity and cracks can compromise mechanical properties and overall performance. This study focuses on in-situ monitoring and characterization of melt pools associated with porosity, aiming to improve defect detection and quality control in DED-printed parts. Traditional machine learning approaches for defect identification rely on extensive labeled datasets, often scarce and expensive to generate in real-world manufacturing. To address this, our framework employs self-supervised learning on unlabeled melt pool data using a Vision Transformer-based Masked Autoencoder (MAE) to produce highly representative embeddings. These fine-tuned embeddings are leveraged via transfer learning to train classifiers on a limited labeled dataset, enabling the effective identification of melt pool anomalies. We evaluate two classifiers: (1) a Vision Transformer (ViT) classifier utilizing the fine-tuned MAE Encoder's parameters and (2) the fine-tuned MAE Encoder combined with an MLP classifier head. Our framework achieves overall accuracy ranging from 95.44% to 99.17% and an average F1 score exceeding 80%, with the ViT Classifier slightly outperforming the MAE Encoder Classifier. This demonstrates the scalability and cost-effectiveness of our approach for automated quality control in DED, effectively detecting defects with minimal labeled data.

Paper Structure

This paper contains 16 sections, 12 equations, 7 figures, 6 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Experimentation & Data collection
  3. Methodology
  4. Masked AutoEncoder (MAE)
  5. MAE Encoder
  6. MAE Decoder
  7. Self-supervised Framework with classifiers
  8. Classifier Models
  9. ViT Classifier
  10. MAE Encoder classifier
  11. Results & Discussion
  12. Training setup of self-supervised MAE
  13. Training setup of supervised Classifiers
  14. Performance Evaluation
  15. Limitations & Future Work
  16. ...and 1 more sections

Figures (7)

  • Figure 1: Cross-sectional view of Directed Energy Deposition (DED) processes redrawn from era2023machine: (a)Powder-feed and (b)Wire-feed.
  • Figure 2: (a) The pyrometer setup inside the DED printing chamber; (b) Printed Samples.
  • Figure 3: Examples of (a) A normal melt pool; (b) An abnormal melt pool.
  • Figure 4: Overview of the framework for in-situ characterization of melt pool images using a self-supervised MAE with classifier models.
  • Figure 5: Overview of Vision Transformer Classifier redrawn from dosovitskiy2020image; (a) Transformer Encoder Architecture, (b) MLP head Architecture.
  • ...and 2 more figures