Noise Calibration and Spatial-Frequency Interactive Network for STEM Image Enhancement

TL;DR

This work tackles the challenge of noisy STEM imaging by introducing a calibrated noise model and a realistic data-synthesis pipeline that captures both HAADF and BF modes and a range of atomic configurations. It then proposes the Spatial-Frequency Interactive Network (SFIN), which integrates frequency-domain processing via a Frequency Convolution Module with cross-domain interactions to exploit the periodic nature of atomic arrangements. Key contributions include a STEM noise calibration method for background, scan, and pointwise noise; a diversified synthetic dataset with random defects and imaging modes; and a network that delivers superior enhancement performance and benefits to atom detection and super-resolution. The approach improves the realism of synthetic STEM data and demonstrates practical impact for atomic-scale imaging tasks and analysis.

Abstract

Scanning Transmission Electron Microscopy (STEM) enables the observation of atomic arrangements at sub-angstrom resolution, allowing for atomically resolved analysis of the physical and chemical properties of materials. However, due to the effects of noise, electron beam damage, sample thickness, etc, obtaining satisfactory atomic-level images is often challenging. Enhancing STEM images can reveal clearer structural details of materials. Nonetheless, existing STEM image enhancement methods usually overlook unique features in the frequency domain, and existing datasets lack realism and generality. To resolve these issues, in this paper, we develop noise calibration, data synthesis, and enhancement methods for STEM images. We first present a STEM noise calibration method, which is used to synthesize more realistic STEM images. The parameters of background noise, scan noise, and pointwise noise are obtained by statistical analysis and fitting of real STEM images containing atoms. Then we use these parameters to develop a more general dataset that considers both regular and random atomic arrangements and includes both HAADF and BF mode images. Finally, we design a spatial-frequency interactive network for STEM image enhancement, which can explore the information in the frequency domain formed by the periodicity of atomic arrangement. Experimental results show that our data is closer to real STEM images and achieves better enhancement performances together with our network. Code will be available at https://github.com/HeasonLee/SFIN}{https://github.com/HeasonLee/SFIN.

Figures (11)

• Figure 1: The three efforts we undertake to boost the performance of STEM image enhancement. (1) Calibrating the noise in real STEM images to make synthetic data more realistic; (2) Increasing the generalization of the dataset by adding randomized atomic arrangements and BF mode images; (3) Incorporating frequency domain operations in the image enhancement network to leverage the regularities in atomic arrangements.
• Figure 2: Our motivation for using frequency domain in STEM image enhancement. When atoms are arranged periodically, simple filtering of the frequency domain image corresponding to the original image can achieve a good denoising effect.
• Figure 3: The process of calibrating scan noise and pointwise noise in our STEM noise calibration method. The light red rectangular area in subfigure (e) indicates fluctuations caused by limited data, which will be discarded during fitting.
• Figure 4: Our dataset simulates different atomic arrangements, including periodic patterns, atom embeddings, atom defects, and random arrangements. This enables our dataset to handle more situations. Other datasets such as GAN GAN and TEMImageNet AtomSegNet do not include atom embeddings and random arrangements.
• Figure 5: The framework of our Spatial-Frequency Interactive Network (SFIN) and the Spatial-Frequency Interactive Block (SFIB) and Frequency Convolution Module (FCM) used in it. Each spatial-frequency interactive block processes spatial and frequency domain features interactively. BN means batch normalization and IFFT means inverse fast Fourier transform.