Deep generative priors for robust and efficient electron ptychography

Abstract

Electron ptychography enables dose-efficient atomic-resolution imaging, but conventional reconstruction algorithms suffer from noise sensitivity, slow convergence, and extensive manual hyperparameter tuning for regularization, especially in three-dimensional multislice reconstructions. We introduce a deep generative prior (DGP) framework for electron ptychography that uses the implicit regularization of convolutional neural networks to address these challenges. Two DGPs parameterize the complex-valued sample and probe within an automatic-differentiation mixed-state multislice forward model. Compared to pixel-based reconstructions, DGPs offer four key advantages: (i) greater noise robustness and improved information limits at low dose; (ii) markedly faster convergence, especially at low spatial frequencies; (iii) improved depth regularization; and (iv) minimal user-specified regularization. The DGP framework promotes spatial coherence and suppresses high-frequency noise without extensive tuning, and a pre-training strategy stabilizes reconstructions. Our results establish DGP-enabled ptychography as a robust approach that reduces expertise barriers and computational cost, delivering robust, high-resolution imaging across diverse materials and biological systems.

Figures (11)

• Figure 1: Overview of the DGP-enabled ptychographic reconstruction algorithm.a Two DGPs generate the sample object and probe used in a mixed-state multislice ptychography forward model. Gradients are computed via automatic differentiation and backpropagated to simultaneously update both DGPs and any auxiliary parameters. b Randomly instantiated DGPs with noise-only inputs often yield unstable reconstructions and rarely converge to a physical solution. c The DGPs can be pre-trained as autoencoders using an approximate object and probe obtained from a conventional iterative or single-shot reconstruction. d The pre-trained DGPs are then employed within the full ptychographic forward model to produce stable reconstructions.
• Figure 1: Information limit for the MOSS-6 reconstructions.a Power spectrum of the reconstructed object using a DGP to generate the object and probe. This is the same data as in main text Fig. \ref{['fig:MOSS6']}b, but shown here with a lower contrast range to emphasize the weaker, high-frequency signal. b Magnified view of the red boxed region in a, with a circle identifying a peak at 1.57. c Same as a but for the pixelated reconstruction as shown in Fig. \ref{['fig:MOSS6']}d. d The identified peak from c showing an information limit of 1.98.
• Figure 2: Denoised reconstructions of the MOSS-6 MOF.a Reconstruction using a DGP to generate both the sample object and probe with maximal resolution and minimal noise. b FFT of a. c, d Reconstruction and FFT of the same dataset using a pixelated object and probe. e, f Reconstruction and FFT using a DGP-generated object and a pixelated probe. g, h Reconstruction and FFT using a pixelated object and DGP-generated probe.
• Figure 2: Reconstructing a simulated gold nanoparticle.a The ground truth phase shift from a simulated gold nanoparticle on amorphous carbon. b--d Snapshots of the reconstruction using a pixelated object and probe, after 10, 91, and 455 respectively. e--g Snapshots of the reconstructed object using a DGP-generated object and probe, after 10, 22, and 57 respectively. h, i Line plots across the dashed line in b showing the convergence of the bulk nanoparticle phase shift for the pixelated and DGP reconstructions.
• Figure 3: Accelerating reconstructions of Au nanoparticles.a--c The reconstructed object after 10, 96, and 2380 when using a pixelated object and probe. d Line scan across the dashed line in a showing the slow reconstruction of the bulk nanoparticle phase shift. e--h The same dataset reconstructed using a DGP for the object and probe. i Magnified view of the red box in c. j FFT of i, inset shows the background subtracted radial profile of the FFT. k,l Magnified view and FFT of the orange box in g. m Plotting the structural similarity vs time for pixelated and DGP reconstructions of a simulated Au nanoparticle dataset.