Neural rendering enables dynamic tomography

TL;DR

This work proposes that neural rendering tools can be used to drive the paradigm shift to enable 3d reconstruction during dynamic events and develops a spatio-temporal model with spline-based deformation field that can reconstruct the spatio-temporal deformation of lattice samples in real-world experiments.

Abstract

Interrupted X-ray computed tomography (X-CT) has been the common way to observe the deformation of materials during an experiment. While this approach is effective for quasi-static experiments, it has never been possible to reconstruct a full 3d tomography during a dynamic experiment which cannot be interrupted. In this work, we propose that neural rendering tools can be used to drive the paradigm shift to enable 3d reconstruction during dynamic events. First, we derive theoretical results to support the selection of projections angles. Via a combination of synthetic and experimental data, we demonstrate that neural radiance fields can reconstruct data modalities of interest more efficiently than conventional reconstruction methods. Finally, we develop a spatio-temporal model with spline-based deformation field and demonstrate that such model can reconstruct the spatio-temporal deformation of lattice samples in real-world experiments.

Figures (14)

• Figure 1: (a) X-ray CT projection geometry. Two projections are illustrated, $\mathcal{X}_1,\mathcal{X}_2$, separated by angle $\alpha_{12}$. The global reference frame, $(x,y,z)$, and on-screen coordinate frames $(r_i,s_i)$ are indicated. (b) Illustration of the framework. Projection data of two types are obtained: at times ${t}=0$ and ${t}=1$, rich projection data is obtained, while during the dynamic experiment at intermediate ${t}$ sparse few-projection data is recorded. In pre-training steps, rich many-projection data are reconstructed using computed tomography and the canonical volume is fitted (dashed lines). Subsequently, the deformation field is calculated from the sparse few-projection data. Full field spatio-temporal information can then be obtained from the two fields.
• Figure 2: (a) Theoretical result for the mutual information between two projections separated by angle $\alpha$. Thin grey lines correspond to samples drawn from various Gaussian distributions while the thick orange line is the mutual information for an axisymmetric Gaussian. (b) Reconstruction error plotted as $1-\mathcal{C}_{cor}$ as a function of separation angle $\alpha$ between 2 projections which are used to train the model for two simulated datasets (balls and pillars).
• Figure 3: Overview of the data efficiency study. Each column corresponds to one dataset. An $xy-$plane is highlighted in the 3d view which corresponds to the slicing plane shown in the subsequent rows: ground truth, NeRF trained with 128 and 3 projections, SIRT (conventional reconstruction) trained with 128 and 3 projections.
• Figure 4: (a) Convergence of normalized correlation $\mathcal{C}_{cor}$ with number of projections for various datasets. Our neural rendering technique is more efficient up to approx. 100 projections. (b) Convergence of normalized correlation $\mathcal{C}_{cor}$ with number of projections for various datasets using experimental data.
• Figure 5: Verification of mutual information theoretical result with three projections. (a) Analytical contours of the mutual information for three projections. (b,c) Contours of reconstruction error proxy, $1-\mathcal{C}_{cor}$, for three projections in simulated datasets: balls (b) and pillars (c).