Accelerated deep self-supervised ptycho-laminography for three-dimensional nanoscale imaging of integrated circuits

TL;DR

This work tackles the challenge of 3D nanoscale imaging of integrated circuits via ptycho-laminography by introducing ADePt, a physics-regularized, self-supervised learning framework. The method integrates a lightweight pre-processor with a deep image prior–based network to reconstruct 3D IC morphology from severely undersampled projections, achieving up to $16\times$ fewer angular samples and $4.67\times$ faster computation while maintaining or surpassing the quality of densely sampled reconstructions due to effective missing-cone filling. Key contributions include a concrete architecture with a forward model driven loss, a two-stage pre-processing and optimization pipeline, and quantitative evidence that the approach outperform baseline methods under sparse sampling. The results indicate significant practical impact for rapid, high-resolution non-destructive IC inspection, reducing data acquisition and compute time without sacrificing fidelity.

Abstract

Three-dimensional inspection of nanostructures such as integrated circuits is important for security and reliability assurance. Two scanning operations are required: ptychographic to recover the complex transmissivity of the specimen; and rotation of the specimen to acquire multiple projections covering the 3D spatial frequency domain. Two types of rotational scanning are possible: tomographic and laminographic. For flat, extended samples, for which the full 180 degree coverage is not possible, the latter is preferable because it provides better coverage of the 3D spatial frequency domain compared to limited-angle tomography. It is also because the amount of attenuation through the sample is approximately the same for all projections. However, both techniques are time consuming because of extensive acquisition and computation time. Here, we demonstrate the acceleration of ptycho-laminographic reconstruction of integrated circuits with 16-times fewer angular samples and 4.67-times faster computation by using a physics-regularized deep self-supervised learning architecture. We check the fidelity of our reconstruction against a densely sampled reconstruction that uses full scanning and no learning. As already reported elsewhere [Zhou and Horstmeyer, Opt. Express, 28(9), pp. 12872-12896], we observe improvement of reconstruction quality even over the densely sampled reconstruction, due to the ability of the self-supervised learning kernel to fill the missing cone.

Figures (5)

• Figure 1: Accelerated deep self-supervised ptycho-laminography. (A) Ptycho-laminographic imaging geometry. Synchrotron X-rays illuminate a sample of integrated circuits in the ptycho-laminography geometry, with the sample rotating around the oblique laminographic axis and scanned over a few thousand angles. For each ptychography scan, the sample is laterally scanned at several hundred different locations. (B) Equivalent imaging geometry. Forward operators $H_n\:(n=1,2,\cdots,N)$ are defined according to each laminographic rotation. (C) Proposed physics-informed machine learning framework. Our pre-processor translates experimental ptycho-laminographic measurement from the detector plane to the sample domain with minimal processing using a ptychographic reconstruction algorithm. ADePt generates a three-dimensional image of integrated circuit morphology from the pre-processed projections throughout the optimization process. (D) Deep neural network architecture for self-supervised learning. The proposed architecture is essentially an encoder-decoder convolutional neural network with skip connections and receiving random noise as input. The output is the image. Code is publicly available at https://github.com/iksungk/ADePt.
• Figure 2: Comparison between FBP & ADePt's reconstructions and the densely sampled reconstruction. (A) We exploit physics-informed machine learning to reliably reconstruct integrated circuits with a reduced number of projections, i.e.$125$ and $250$ out of $2000$, and qualitatively compare the FBP & ADePt's reconstructions with the densely-sampled reconstruction. Please note that the colormaps used in some reconstructions are selected differently. Specifically, for (1) the densely-sampled reconstruction, the colormaps of Layers 2-5 are set to its 2.5th and 80th percentiles, and for (2) the FBP reconstructions with both 125 and 250 angles, the colormaps of Layers 2-5 are fixed to its 12.5th and 80th percentiles. For all other figures, the colormaps are fixed to the minimum and maximum values of their respective reconstructions. (B) We qualitatively compare the reconstructions within their respective zoomed-in areas for better evaluation. The colormap conventions followed in (A) are also applied.
• Figure 3: Qualitative comparison among $yz$ cross-sections of different reconstructions. Although the densely sampled reconstruction yields the best contrast among coarser features, missing cone artifacts are not completely addressed (see red arrows). Some finer features can be displayed more clearly with ADePt than with the FBP reconstructions and the densely-sampled reconstruction, at the cost of quantitativeness (see yellow boxes).
• Figure 4: Power spectral density analysis for qualitative comparison. We visualize FBP & ADePt's reconstructions and the densely sampled reconstruction in $k$-space to visualize artifacts due to missing cone and sparse sampling. Cuts are made along the $k_y\hbox{-}k_z$ and $k_x\hbox{-}k_y$ planes (red arrows). We demonstrate that ADePt provides reconstructions with fewer artifacts in $k$-space, considering that the FBP reconstructions display artifacts due to the angular subsampling and missing cone, and that the densely-sampled reconstruction shows artifacts due to imperfect missing cone filling (black arrows).
• Figure 5: Quantitative analysis of reconstructions and ablation study. (A) We use bit-error rate (BER) and Pearson correlation coefficient (PCC) to compare ADePt's reconstructions with the baseline for different scales of features in the integrated circuits. (B) Ablation study. We assess relative contribution of each design element, i.e. high-pass filtering (HPF) and total-variation (TV) regularization, to the final reconstruction by removing one at a time from the complete model. We incorporate HPF to enforce a high-frequency content bias to our deep neural network and TV regularization to suppress spurious high-frequency artifacts in the background. Fig. S4 in Supplementary Materials illustrates (A) and (B) further using another quantitative metric. (C) The self-supervised learning algorithm behaves unfavorably when each component is ablated. Total variation (TV) regularization suppresses high-frequency artifacts, and high-pass filtering (HPF) improves the spatial resolution of features recovered by the algorithm (red arrows).