Deep-Learning Recognition of Scanning Transmission Electron Microscopy: Quantifying and Mitigating the Influence of Gaussian Noises

TL;DR

A deep-learning mask region-based neural network (Mask R-CNN) for the recognition of nanoparticles imaged by STEM, as well as generating the associated dimensional analysis, which yields satisfying accuracy compared with the traditional threshold methods.

Abstract

Scanning transmission electron microscopy (STEM) is a powerful tool to reveal the morphologies and structures of materials, thereby attracting intensive interests from the scientific and industrial communities. The outstanding spatial (atomic level) and temporal (ms level) resolutions of the STEM techniques generate fruitful amounts of high-definition data, thereby enabling the high-volume and high-speed analysis of materials. On the other hand, processing of the big dataset generated by STEM is time-consuming and beyond the capability of human-based manual work, which urgently calls for computer-based automation. In this work, we present a deep-learning mask region-based neural network (Mask R-CNN) for the recognition of nanoparticles imaged by STEM, as well as generating the associated dimensional analysis. The Mask R-CNN model was tested on simulated STEM-HAADF results with different Gaussian noises, particle shapes and particle sizes, and the results indicated that Gaussian noise has determining influence on the accuracy of recognition. By applying Gaussian and Non-Local Means filters on the noise-containing STEM-HAADF results, the influences of noises are largely mitigated, and recognition accuracy is significantly improved. This filtering-recognition approach was further applied to experimental STEM-HAADF results, which yields satisfying accuracy compared with the traditional threshold methods. The deep-learning-based method developed in this work has great potentials in analysis of the complicated structures and large data generated by STEM-HAADF.

Figures (7)

• Figure 1: Flowchart demonstrating the design principles of the deep-learning-based models for recognition of nanoparticles from STEM-HAADF images. A Mask R-CNN based model and a threshold-based method were adopted for the nanoparticle recognition, and experimental and simulated STEM-HAADF images were loaded into the two models. The models output results of feature recognition, and the results were evaluated for accuracy. The evaluations were then utilized for refining the models selected in the first step.
• Figure 2: Evaluating the influence of morphological and microscopic factors on the accuracy of nanoparticle recognition. Two recognition models, threshold-based and deep-learning-based, were adopted for imaging processing and nanoparticle recognition. Both simulated and experimental STEM-HAADF images were input into the models for testing. The influence of gaussian noise, particle shape and particle size on the accuracy of recognition were evaluated.
• Figure 3: Influences of Gaussian noises on the recognition accuracy of STEM-HAADF nanospheres. (a-c) Three different levels of Gaussian noise were introduced into the simulated STEM-HAADF images, namely no noise, SNR = 14.5497 and SNR = 7.4870. The simulated STEM-HAADF images with different Gaussian noises are presented in the first column, with their authentic locations are presented in the second column. The Mask R-CNN and threshold-based recognition results are presented in the third and fourth columns for comparison, respectively. Size distributions generated from the Mask R-CNN and threshold-based results are presented in the last column. (d) Intensity Profiles of the three input STEM-HAADF images with different levels of Gaussian noise, as presented in (a).
• Figure 4: Influences of particle shape on the recognition accuracy. (a-c) Simulated STEM-HAADF image of nanospheres and the corresponding Mask R-CNN recognition result. (d) Simulated STEM-HAADF image of nanorods and the corresponding Mask R-CNN testing. (e-g) Simulated STEM-HAADF image of concave nanocubes and the corresponding Mask R-CNN recognition result.
• Figure 5: Influences of particle size on the recognition accuracy. (a-i) Simulated STEM-HAADF image of 30-, 50- and 70-pixel nanospheres, the corresponding Mask R-CNN recognition results, and representative zoom-in views.