Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MatSAM: Efficient Extraction of Microstructures of Materials via Visual Large Model

Changtai Li, Xu Han, Chao Yao, Xiaojuan Ban

TL;DR

MatSAM tackles the challenge of extracting material microstructures from micograph images without manual annotations by adapting the Segment Anything Model (SAM) with a structure-aware, unsupervised prompt strategy. It combines coarse, rule-based pre-segmentation with adaptive grid prompt placement to guide SAM, enabling zero-shot, training-free segmentation of grains, phases, and defects across diverse OM/SEM datasets. Across 16 datasets, including polycrystalline and multiphase micrographs, MatSAM outperforms conventional methods and rivals supervised approaches, while on several public datasets it matches or exceeds specialist models under zero-shot conditions. This approach promises a scalable, annotation-light pathway for rapid quantitative microstructure characterization, accelerating materials design and property optimization workflows.

Abstract

Efficient and accurate extraction of microstructures in micrographs of materials is essential in process optimization and the exploration of structure-property relationships. Deep learning-based image segmentation techniques that rely on manual annotation are laborious and time-consuming and hardly meet the demand for model transferability and generalization on various source images. Segment Anything Model (SAM), a large visual model with powerful deep feature representation and zero-shot generalization capabilities, has provided new solutions for image segmentation. In this paper, we propose MatSAM, a general and efficient microstructure extraction solution based on SAM. A simple yet effective point-based prompt generation strategy is designed, grounded on the distribution and shape of microstructures. Specifically, in an unsupervised and training-free way, it adaptively generates prompt points for different microscopy images, fuses the centroid points of the coarsely extracted region of interest (ROI) and native grid points, and integrates corresponding post-processing operations for quantitative characterization of microstructures of materials. For common microstructures including grain boundary and multiple phases, MatSAM achieves superior zero-shot segmentation performance to conventional rule-based methods and is even preferable to supervised learning methods evaluated on 16 microscopy datasets whose micrographs are imaged by the optical microscope (OM) and scanning electron microscope (SEM). Especially, on 4 public datasets, MatSAM shows unexpected competitive segmentation performance against their specialist models. We believe that, without the need for human labeling, MatSAM can significantly reduce the cost of quantitative characterization and statistical analysis of extensive microstructures of materials, and thus accelerate the design of new materials.

MatSAM: Efficient Extraction of Microstructures of Materials via Visual Large Model

TL;DR

MatSAM tackles the challenge of extracting material microstructures from micograph images without manual annotations by adapting the Segment Anything Model (SAM) with a structure-aware, unsupervised prompt strategy. It combines coarse, rule-based pre-segmentation with adaptive grid prompt placement to guide SAM, enabling zero-shot, training-free segmentation of grains, phases, and defects across diverse OM/SEM datasets. Across 16 datasets, including polycrystalline and multiphase micrographs, MatSAM outperforms conventional methods and rivals supervised approaches, while on several public datasets it matches or exceeds specialist models under zero-shot conditions. This approach promises a scalable, annotation-light pathway for rapid quantitative microstructure characterization, accelerating materials design and property optimization workflows.

Abstract

Efficient and accurate extraction of microstructures in micrographs of materials is essential in process optimization and the exploration of structure-property relationships. Deep learning-based image segmentation techniques that rely on manual annotation are laborious and time-consuming and hardly meet the demand for model transferability and generalization on various source images. Segment Anything Model (SAM), a large visual model with powerful deep feature representation and zero-shot generalization capabilities, has provided new solutions for image segmentation. In this paper, we propose MatSAM, a general and efficient microstructure extraction solution based on SAM. A simple yet effective point-based prompt generation strategy is designed, grounded on the distribution and shape of microstructures. Specifically, in an unsupervised and training-free way, it adaptively generates prompt points for different microscopy images, fuses the centroid points of the coarsely extracted region of interest (ROI) and native grid points, and integrates corresponding post-processing operations for quantitative characterization of microstructures of materials. For common microstructures including grain boundary and multiple phases, MatSAM achieves superior zero-shot segmentation performance to conventional rule-based methods and is even preferable to supervised learning methods evaluated on 16 microscopy datasets whose micrographs are imaged by the optical microscope (OM) and scanning electron microscope (SEM). Especially, on 4 public datasets, MatSAM shows unexpected competitive segmentation performance against their specialist models. We believe that, without the need for human labeling, MatSAM can significantly reduce the cost of quantitative characterization and statistical analysis of extensive microstructures of materials, and thus accelerate the design of new materials.
Paper Structure (13 sections, 3 equations, 8 figures, 5 tables)

This paper contains 13 sections, 3 equations, 8 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Results
  3. MatSAM: Materials Segment Anything Model
  4. Material micrograph collection and evaluation
  5. Polycrystalline image segmentation
  6. Multiphase image segmentation
  7. Evaluating MatSAM on public datasets
  8. Discussion
  9. Methods
  10. Datasets and Evaluation metrics
  11. Main architecture of MatSAM
  12. Structure-aware prompt point generation
  13. Implementation details

Figures (8)

  • Figure 1: Examples of SAM image segmentation results and the overall architecture of MatSAM. (a) The segmentation results of the native SAM on natural scene images. (b) The segmentation results of the native SAM on three types of material microscopy images. (c) The processing flow and results of MatSAM for material microscopy images.
  • Figure 2: The materials microscopy image dataset used in this paper. The dataset is composed of metal materials, including pure iron, stainless steel, duplex steel, special steel, high-temperature alloy, and high-entropy alloy. The materials are classified into single-phase polycrystalline, multiphase polycrystalline, single-phase polycrystalline, and defects based on the structural features. The imaging methods are divided into optical microscopy and scanning electron microscopy. Best viewed in color.
  • Figure 3: A comparison of MatSAM with conventional rule-based methods on a multi-crystal dataset. The first column shows the original input image, the second column shows the ground truth and the third to sixth columns show the results of MatSAM, OTSU, Canny, and Watershed, respectively. The corresponding segmentation index ARI ARI is labeled in each result figure, with higher values indicating better performance.
  • Figure 4: Example of MatSAM versus conventional rule-based methods in a multiphase dataset. The first column shows the original input image, the second column shows the manual annotation, and the third to sixth columns show the output results of MatSAM, OTSU, Canny, and Watershed, respectively. The corresponding segmentation metrics IoU are marked in the results of MatSAM and OTSU (the higher the metric, the better). Since the segmentation results of Canny and Watershed methods are difficult to form effective closed regions, the specific IoU values are not marked.
  • Figure 5: Examples of extracting crucial microstructures in multiphase images. The first row is raw images: (a) NBS-1, (b) NBS-1 (cropped), (c) IN939-1, and (d) IN939-2 (with cracks). The second row is the segmentation masks overlaying the raw images for better visualization. The third row is the binary images highlighting different phases (foreground). Best viewed in color.
  • ...and 3 more figures