Efficient Semi-Automated Material Microstructure Analysis Using Deep Learning: A Case Study in Additive Manufacturing

Abstract

Image segmentation is fundamental to microstructural analysis for defect identification and structure-property correlation, yet remains challenging due to pronounced heterogeneity in materials images arising from varied processing and testing conditions. Conventional image processing techniques often fail to capture such complex features rendering them ineffective for large-scale analysis. Even deep learning approaches struggle to generalize across heterogeneous datasets due to scarcity of high-quality labeled data. Consequently, segmentation workflows often rely on manual expert-driven annotations which are labor intensive and difficult to scale. Using an additive manufacturing (AM) dataset as a case study, we present a semi-automated active learning based segmentation pipeline that integrates a U-Net based convolutional neural network with an interactive user annotation and correction interface and a representative core-set image selection strategy. The active learning workflow iteratively updates the model by incorporating user corrected segmentations into the training pool while the core-set strategy identifies representative images for annotation. Three subset selection strategies, manual selection, uncertainty driven sampling and proposed maximin Latin hypercube sampling from embeddings (SMILE) method were evaluated over six refinement rounds. The SMILE strategy consistently outperformed other approaches, improving the macro F1 score from 0.74 to 0.93 while reducing manual annotation time by about 65 percent. The segmented defect regions were further analyzed using a coupled classification model to categorize defects based on microstructural characteristics and map them to corresponding AM process parameters. The proposed framework reduces labeling effort while maintaining scalability and robustness and is broadly applicable to image based analysis across diverse materials systems.

Figures (4)

• Figure 1: Semi-automated pipeline for microstructure image analysis. The workflow consists of two stages: defect detection and defect classification. In the defect detection stage, an active learning framework combined with a subset selection strategy and a user-friendly annotation interface is employed to segment regions of microstructural images containing defects. The subset selection strategy ensures the use of diverse and representative images, active learning enables iterative improvement of the detection model, and the annotation tool allows users to efficiently correct model-predicted labels rather than generating ground-truth annotations from scratch. Subsequently, in the defect classification stage, the obtained segmented defect regions are mapped to corresponding etched microstructural images, containing critical microstructural features such as melt pool boundaries and grain morphology. This combined information is used to train a defect classification model, and the resulting defect classification information is mapped to associated manufacturing process parameters.
• Figure 2: Defect detection model performance. (a) Iterative improvement in the macro F1 score of the defect detection model with increasing active learning rounds for different subset selection strategies. Results obtained using the baseline Otsu thresholding method are also shown for comparison. Markers denote the mean performance on the test set, and error bars indicate $\pm1\sigma$ standard deviation. The dashed line and shaded region represent the mean macro F1 score and $\pm1\sigma$ standard deviation of the Otsu method, respectively. (b) Example segmentation results produced by the final best-performing SMILES-based model on test images.
• Figure 3: Diversity and coverage of subset selection strategies and Representative Grad-CAM visualization using the final SMILE-trained model. (a) Isomap projection of the full training and the independent test set, highlighting image samples selected using manual, deep ensemble, and SMILE-based approaches across all active learning iterations. Regions that are either underrepresented or overrepresented by different selection strategies are also highlighted. (b) t-SNE projection of the image samples selected using the SMILE approach. While selection of points across various clusters promotes coverage of the embedding space, the LHS + maximin criteria enforces intra-cluster diversity. (c) The left panel shows the original microstructural image, while the right panel displays the Grad-CAM heatmap overlaid on the image. High activation regions are localized around defect areas, confirming that the network focuses on physically meaningful microstructural features rather than background intensity variations.
• Figure 4: Defect classification model performance and defect statistics measured across different AM process parameters. (a) Confusion matrix summarizing performance of the defect classification model on the test set. (b) Representative test patches comparing ground-truth and predicted labels, illustrating both correct and incorrect classifications. Defect statistics measured across different laser power and scan speed combinations for (c) Inconel 625 and (d) CoCrMo alloy AM systems. Each cell represents one processing condition and contains two bar plots, one capturing relative fraction of lack of fusion (blue) and porosity (orange), with the white colored number indicating the porosity fraction, and second bar denoting % defect area fraction (red color) with the average number of defects reported in brackets. The results highlight clear variations in defect type and severity across processing conditions, demonstrating the strong influence of process parameters on defect formation.