Atomic Depth Estimation From Noisy Electron Microscopy Data Via Deep Learning

TL;DR

This work tackles the challenge of recovering 3D atomic-scale depth from noisy TEM images by reframing depth estimation as per-pixel semantic segmentation. It introduces SegDepth, a UNet-based framework trained on synthetic, Poisson-noised TEM data to produce per-pixel depth distributions over $d \in \{0,\dots,10\}$ with an associated confidence score. Key contributions include a complete data-generation pipeline for CeO2 nanoparticles, a segmentation-based depth estimator with calibration and gradient-based interpretability, and qualitative validation on real TEM data. The approach enables robust, spatially resolved tracking of atomic-column depth and dynamics in dynamic TEM studies, with implications for catalysis and materials science.

Abstract

We present a novel approach for extracting 3D atomic-level information from transmission electron microscopy (TEM) images affected by significant noise. The approach is based on formulating depth estimation as a semantic segmentation problem. We address the resulting segmentation problem by training a deep convolutional neural network to generate pixel-wise depth segmentation maps using simulated data corrupted by synthetic noise. The proposed method was applied to estimate the depth of atomic columns in CeO2 nanoparticles from simulated images and real-world TEM data. Our experiments show that the resulting depth estimates are accurate, calibrated and robust to noise.

Figures (12)

• Figure 1: Depth estimation via segmentation. (a) Atomic-column depth of a CeO2 nanoparticle in (110) zone axis orientation. In this orientation the Ce and O atomic columns are distinct. The thickness of the material increases from top to bottom. (b) Noisy data generated by corrupting the TEM image corresponding to (a) with Poisson noise to simulate real acquisition data. The Ce columns in the thinner columns at the top are bright and have fainter oxygen columns to the left and right. In the thicker columns at the bottom the contrast is reversed. (c) Estimate of the atomic-column depth obtained by the proposed deep-learning based method, where a neural network is trained to approximate the depth profile (a) from the noisy data (c). (d) Confidence score of the proposed model at each pixel, computed as defined in \ref{['eq:confidence']}, which quantifies the uncertainty in the model estimates. The confidence is generally high, except on the boundary of the nanoparticle, where estimation is more challenging.
• Figure 2: The SegDepth framework.Top: A deep convolutional neural network is trained to estimate atomic-column depth using simulated data. Different 3D CeO2 nanoparticle structures are used to generate simulated TEM data, which are then corrupted with Poisson noise to produce the noisy input to the network. The 3D structure is also used to create segmentation maps that encode the atomic-column depth. The network is trained to approximate these segmentation maps. Bottom: At inference, noisy TEM data is fed into the neural network to produce a segmentation map that estimates the atomic-column depth at every pixel.
• Figure 3: Data simulation. Examples of simulated depth masks (left) and their associated TEM images before (center) and after (right) corrupting them with Poisson noise.
• Figure 4: Spatial label smoothing. (a) The pixel-wise depth segmentation labels obtained by projecting the nanoparticle atomic structure onto the imaging plane, as described in Section \ref{['sec:depth_segmentation_maps']}, contain irregular noisy patterns that change abruptly. (b) Spatial smoothing, so that the labels vary in concentric rings around each atomic column, produces more regular patterns that are easier to learn.
• Figure 5: Spatial weighting to address class imbalance. We utilize a spatial weight within the training loss in \ref{['eq:weighted_ce']} to penalize misclassification in pixels that are closer to the atomic column centers. This mitigates class imbalance due to the abundance of background pixels that do not contain atomic columns.