Differentiable Electron Microscopy Simulation: Methods and Applications for Visualization

TL;DR

The paper introduces DiffTEM, a differentiable TEM simulator extended with GPU-accelerated forward rendering (gputem) and a learning-enabled differentiable module (difftem). It enables automatic detector parameter (MTF) estimation and physics-based denoising of real cryo-EM data, by backpropagating through the imaging chain and employing surrogate loss frameworks for stochastic components. The approach yields MTf estimates that match ground-truth detector behavior and denoised micrographs that surpass state-of-the-art methods, improving downstream tomographic reconstruction and visualization. This scalable, physics-grounded framework supports solving inverse problems in electron microscopy and offers a path toward integrating neural components while preserving physical interpretability.

Abstract

We propose a new microscopy simulation system that can depict atomistic models in a micrograph visual style, similar to results of physical electron microscopy imaging. This system is scalable, able to represent simulation of electron microscopy of tens of viral particles and synthesizes the image faster than previous methods. On top of that, the simulator is differentiable, both its deterministic as well as stochastic stages that form signal and noise representations in the micrograph. This notable property has the capability for solving inverse problems by means of optimization and thus allows for generation of microscopy simulations using the parameter settings estimated from real data. We demonstrate this learning capability through two applications: (1) estimating the parameters of the modulation transfer function defining the detector properties of the simulated and real micrographs, and (2) denoising the real data based on parameters trained from the simulated examples. While current simulators do not support any parameter estimation due to their forward design, we show that the results obtained using estimated parameters are very similar to the results of real micrographs. Additionally, we evaluate the denoising capabilities of our approach and show that the results showed an improvement over state-of-the-art methods. Denoised micrographs exhibit less noise in the tilt-series tomography reconstructions, ultimately reducing the visual dominance of noise in direct volume rendering of microscopy tomograms.

Figures (15)

• Figure 1: Three main contributions of this work are: (1) DiffTEM: a differentiable electron microscopy simulator, (2) a use case for automatic detector parameter estimation, and (3) a use case for denoising the data from real electron microscope.
• Figure 2: Our gputem simulator receives scene configuration and detector parameters estimated from real data by difftem simulator for generating simulated micrograph. Our difftem simulator denoises the real data, the denoised data can be used for reconstruction and the reconstrution can be visualized by direct volume rendering.
• Figure 3: tem simulator contains an electron guns, optical system with many lens, a detector and a specimen. Each electron is defined by its wave function, it interacts with specimen give the scattered wave function which is recorded at detectors to form micrographs.
• Figure 4: Compute Graph of Detector Parameter Estimation: The difftem first receives noisy training data, computes the loss with simulated noisy data, applies the predicted mtf, and performs gradient descent for optimizing mtf parameters.
• Figure 5: Compute graph of denoising: The projection without mtf is the optimization target. Log likelihood values are used in our surrogate loss function along with the comparison loss that computes the difference between noisy projection predictions and noisy projections in a dataset.