CNN-based TEM image denoising from first principles

TL;DR

This study uses density functional theory calculations with a set of pseudo-atomic orbital basis sets to generate highly accurate ground truth images and introduces four types of noise into these simulations to create realistic training datasets.

Abstract

Transmission electron microscope (TEM) images are often corrupted by noise, hindering their interpretation. To address this issue, we propose a deep learning-based approach using simulated images. Using density functional theory calculations with a set of pseudo-atomic orbital basis sets, we generate highly accurate ground truth images. We introduce four types of noise into these simulations to create realistic training datasets. Each type of noise is then used to train a separate convolutional neural network (CNN) model. Our results show that these CNNs are effective in reducing noise, even when applied to images with different noise levels than those used during training. However, we observe limitations in some cases, particularly in preserving the integrity of circular shapes and avoiding visible artifacts between image patches. To overcome these challenges, we propose alternative training strategies and future research directions. This study provides a valuable framework for training deep learning models for TEM image denoising.

Figures (5)

• Figure 1: (a) Atomic configurations and charge density maps. The red rectangles indicate the regions that differ from each other. (b) Schematic of the process for obtaining the lateral combined charge density map. Here, $n$ represents the number of disordered structures, and $N$ represents the number of lateral combined charge density map images. (c) Scheme for generating shifted images. The magenta rectangle indicates the truncated circle at the edge of the image. (d) Enlarged images illustrating the generated noise in detail. The ground truth image shows one of the lateral combined charge density maps, with the green square indicating the magnified region. The right panel image, labeled BB$+$G, shows the image with applied background brightness and Gaussian noise.
• Figure 2: (a) Enlarged sections of the corrupted and predicted images with $C_{\text{noise}} = 0.2$ and the corresponding MS-SSIM chart. (b) Enlarged sections of the corrupted and predicted images with $C_{\text{noise}} = 0.3$ and the corresponding MS-SSIM chart. The lowest panel shows MS-SSIM values as a function of $C_{\text{noise}}$ for both the interpolation and extrapolation regions.
• Figure 3: Predicted images from experimental data using the models. The red squares show enlarged areas in full-size images. The regions labeled A1, A2, and A3 indicate low-brightness areas in the experimental TEM image. The red arrows point to the bubble-shaped defects generated when predicting extrapolated regions.
• Figure 4: Full-size corrupted images used in the test sets, with the noise coefficient $C_{\text{noise}}$ set to 0.3. The green dashed squares indicate enlarged sections, which are presented in Figure \ref{['fig:fig_model_test']}(b).
• Figure 5: Schematic representation of our workflow.