Noise2Void for Denoising Atomic Resolution Scanning Transmission Electron Microscopy Images

TL;DR

This work extends Noise2Void to a dual-channel, atomic-resolution STEM denoising task by applying it to simultaneously acquired BF and ADF data in graphene liquid cells. The authors introduce architectural, training, and data-handling modifications to enable self-supervised denoising without ground-truth data, achieving substantial improvements in PSNR, SSIM, and N-RMSE over Gaussian and TV baselines. They demonstrate robust performance on experimental data and multislice simulations, with real-time denoising speeds (~45 frames per second) and effective transfer training across datasets. The approach enhances visibility of graphene lattices and Au adatoms, enabling higher-quality, low-dose imaging and faster data analysis in dynamic solid–liquid interfaces with broad applicability to atomic-resolution TEM studies.

Abstract

The Noise2Void technique is demonstrated for successful denoising of atomic-resolution scanning transmission electron microscopy (STEM) images. The technique is applied to denoising atomic resolution images and videos of gold adatoms on a graphene surface within a graphene liquid cell, with the denoised experimental data qualitatively demonstrating improved visibility of both the Au adatoms and the graphene lattice. The denoising performance is quantified by comparison to similar simulated data and the approach is found to significantly outperform both total variation and simple Gaussian blurring. Compared to other denoising methods, the Noise2Void technique has the combined advantages that it requires no manual intervention during training or denoising, no prior knowledge of the sample and is compatible with real time data acquisition rates of at least 45 frames per second.

Figures (17)

• Figure 1: Schematic illustration of the experimental setup and acquired data. (a) An overview of the modified UNet architecture used for Noise2Void denoising where the inputs are noisy dual-channel (HAADF & BF) STEM video frames (left) and the outputs are the same frames denoised (right). ‘Conv2d’, ‘ConvTranspose2d’ and ‘LeakyReLU’ refer to convolutional blocks, transpose-convolution blocks and leaky ‘ReLU’ activation blocks, respectively, while ‘Concat’ and ‘AvgBlurPool’ refer to channel-wise concatenation and bilinear downsampling N2V2, respectively. The black horizontal arrows represent UNet skip connections. (b) The results of a separate atom-finding step applied to the example image above. In the denoised output 89% more adatom features are identified compared to the raw image, showing the improved performance of atom finding analysis when Noise2Void is used in a data-analysis pipeline. Inset are corresponding atomic diagrams of the magnified region, showing the graphene lattice positions with gold adatoms overlain. The blue arrows highlight gold atoms that were only detected after Noise2Void denoising was applied. Also shown is a model of the TEM liquid cell, showing the convergent electron probe (green), the few-layer graphene windows (black) and the boron nitride spacer layer (blue).
• Figure 2: Modified approach to remove chequerboard artifacts found when denoising. (a) Example input dual-channel image. (b) Denoising result showing clear chequerboard artifacts where denoising was performed with an early version of our underlying modified Noise2Void UNet using standard pixel-masking (during training) and transposed convolutions for upsampling. (c) The final Noise2Void UNet architecture, used elsewhere in this work, with chequerboard artifacts significantly diminished. The architectures of (c) and (b) have different upsampling and jittered pixel-masking. An overview of the modified Noise2Void training algorithm can be seen in (d), with the masked pixels jittered/randomly translated by a small amount. Note that, while only annular dark-field images are shown in (d), training is performed on the full dual-channel images.
• Figure 3: Demonstration of Noise2Void denoising performance on the experimental graphene liquid cell data compared to standard methods. Each frame is a dual-channel image (ADF on upper row, BF on lower row). Intensity line-profiles are presented under the images showing the intensity modulations resulting from the crystal lattice of the graphene windows. The line-profiles are taken along the $[10\bar{1}0]$ direction of the graphene lattice, at the same position for all data (indicated by the green and blue arrows on ADF and BF images respectively), with a width of 1 px. From left to right the images compare the same frame a) from the original experimental data used as input for all the denoisers, b) after Gaussian denoising, c) after denoising by the total variation technique and d) after denoising by our modified Noise2Void approach. The square-root of pixel intensities are displayed for the ADF channel, while the BF images and all line-profiles are plotted on a linear scale. Only Noise2Void denoising recovers the periodic graphene lattice both the BF and the ADF channels.
• Figure 4: Demonstration of Noise2Void denoising performance on the experimental graphene liquid cell data (ADF and BF image pairs) and comparison to standard denoising methods. Intensity line-profiles are presented under the images showing the intensity modulations. The line-profiles are taken through two gold adatoms and are at the same position for all data, (indicated by the green and blue arrows on ADF and BF images respectively) with a width of 1 px. From left to right the images compare the same frame a) from the original experimental data used as input for all the denoisers, b) after Gaussian denoising, c) after denoising by the total variation technique and d) after denoising by our modified Noise2Void approach. The square-root of pixel intensities are displayed for the ADF channel, though the BF images and all line-profiles below are plotted linearly. The ADF has a much flatter background with the Noise2Void denoising, compared to the input, Gaussian blur and TV denoising, making it easier to recover the Au adatom positions.
• Figure 5: Fourier space demonstration of Noise2Void denoising performance on the experimental graphene liquid cell data and comparison to standard denoising methods. The magnitudes of 2D fast Fourier transforms (FFTs) of the ADF and BF image channels (first row and second row, respectively) are shown with intensities plotted on a logarithmic scale. The third and fourth rows show a polar transform of the 2D FFTs, for the ADF and BF images respectively, six-way folded in the azimuthal direction with the azimuthally integrated intensity overlaid. For each row, intensities are displayed equally for ease of comparison. From left to right the columns compare the same frame a) from the original experimental data used as input for all the denoisers, b) after Gaussian denoising, c) after denoising by the total variation technique and d) after denoising by our modified Noise2Void approach.