Limited-angle tomographic reconstruction of dense layered objects by dynamical machine learning

TL;DR

This work tackles the ill-posed problem of limited-angle tomography in strongly scattering dense layered objects by reframing multi-view measurements as a dynamical process. It introduces a split-convolutional GRU (SC-GRU) with an encoder/decoder and an angular attention mechanism to iteratively refine 3D refractive-index reconstructions as new angular views arrive. Compared with static priors, the proposed dynamic approach yields fewer artifacts and higher reconstruction fidelity across multiple quantitative metrics, supported by ablation studies that highlight the importance of split convolution and angular attention. The method leverages a beam propagation forward model to generate training data and approximants, enabling robust performance under strong scattering and suggesting broad applicability to other inverse-scattering and multi-view tomography problems. Overall, the dynamic learning framework represents a significant advancement in regularized, data-driven 3D tomography for complex media.

Abstract

Limited-angle tomography of strongly scattering quasi-transparent objects is a challenging, highly ill-posed problem with practical implications in medical and biological imaging, manufacturing, automation, and environmental and food security. Regularizing priors are necessary to reduce artifacts by improving the condition of such problems. Recently, it was shown that one effective way to learn the priors for strongly scattering yet highly structured 3D objects, e.g. layered and Manhattan, is by a static neural network [Goy et al, Proc. Natl. Acad. Sci. 116, 19848-19856 (2019)]. Here, we present a radically different approach where the collection of raw images from multiple angles is viewed analogously to a dynamical system driven by the object-dependent forward scattering operator. The sequence index in angle of illumination plays the role of discrete time in the dynamical system analogy. Thus, the imaging problem turns into a problem of nonlinear system identification, which also suggests dynamical learning as better fit to regularize the reconstructions. We devised a recurrent neural network (RNN) architecture with a novel split-convolutional gated recurrent unit (SC-GRU) as the fundamental building block. Through comprehensive comparison of several quantitative metrics, we show that the dynamic method improves upon previous static approaches with fewer artifacts and better overall reconstruction fidelity.

Figures (7)

• Figure 1: (a) Each angle of illumination, here labelled as angular axis, corresponds to a time step in an analogous temporal axis. (b) The raw intensity diffraction pattern $\mathbf{g}_n,\: n\!=\!1,\ldots,N\!\!=\!\!42$ of the at $n$-th angular sequence step is followed by gradient descent and moving average operations to construct a shorter Approximant sequence $\mathbf{\tilde{f}}_m{}^{[1]},\: m\!=\!1,\ldots,M\!\!=\!\!12$. The Approximants $\mathbf{\tilde{f}}_m{}^{[1]}$ are encoded to $\xi_m$ and fed to the recurrent dynamical operation whose output sequence $\mathbf{h}_m, m\!=\!1,\ldots,12$ the angular attention scheme merges into a single representation $a$, and that is finally decoded to produce the 3D reconstruction $\mathbf{\hat{f}}$. Training adapts the weights of the learned operators in this architecture to minimize the training loss function $\mathcal{E}(\mathbf{f},\hat{\mathbf{f}})$ between $\mathbf{\hat{f}}$ and the ground truth object $\mathbf{f}$.
• Figure 2: Optical apparatus used for experimental data acquisition goy2019high. L$1-4$: lenses, F$1$: pinhole, A$1$: aperture, EM-CCD: electron-multiplying charge coupled device. $f_{\text{L}_3}:f_{\text{L}_4} = 2:1$. The object is rotated along both $x$ and $y$ axes. The defocus distance between the conjugate plane to the exit object surface and the EM-CCD is $\Delta z = 58.2\:\text{mm}$.
• Figure 3: Details on implementing the dynamical scheme of Figure \ref{['fig:introduction']}. (a) Overall network architecture; (b) tensorial dimensions of each layer; (c) down-residual block (DRB); (d) up-residual block (URB); and (e) residual block (RB). $K$ and $S$ indicate the sizes of kernel and stride, respectively, and the values shown apply only to the row and column axes. For the layer axis, $K=4$ and $S=1$ always. The disparities are to implement the split convolution scheme; please see Section \ref{['sec:comput_arch']}.\ref{['subsec:sc_gru']} and Figure \ref{['fig:split_convolution']}.
• Figure 4: Split convolution scheme: different convolution kernels are applied along the lateral $x,y$ axes vs. the longitudinal $z$ axis. In our present implementation, the kernels' respective dimensions are $3 \times 3 \times 1$ (or $1 \times 1 \times 1$) and $1 \times 1 \times 4$. The lateral and longitudinal convolutions are computed separately and the results are then added element-wise. The split convolution scheme is used in both the gated recurrent unit (Section \ref{['sec:comput_arch']}.\ref{['subsec:sc_gru']}) and the encoder/decoder (Section \ref{['sec:comput_arch']}.\ref{['subsec:conv_enc_dec']}).
• Figure 5: Progress of 3D reconstruction performance as new windowed Approximants $m=1,\ldots, M\!\!=\!\!12$ according to (\ref{['eq:moving_window']}) applied on experimental data are presented to the recurrent scheme. The same progression can be found in the Online Materials as a movie.