IMITATE: Image Registration with Context for unknown time frame recovery

TL;DR

The paper addresses artefacts in 4D-CT caused by irregular breathing by proposing IMITATE, a reference-agnostic deformable image registration framework that leverages multiple moving frames and breathing-amplitude conditioning through a conditional U-Net. It formulates a multi-frame, context-aware registration objective with losses that promote similarity, anatomical realism, deformation regularity, and cross-frame consistency, and demonstrates improved frame interpolation and artefact suppression on 4D-CT data with statistical significance. The approach reduces dependence on a fixed reference image and shows potential to lessen the number of acquisitions required for clinically adequate 4D-CT volumes, enabling real-time, artefact-free reconstructions. The work suggests extending conditioning to additional tissue-property signals and integrating elasticity cues for further gains in medical image registration and reconstruction.

Abstract

In this paper, we formulate a novel image registration formalism dedicated to the estimation of unknown condition-related images, based on two or more known images and their associated conditions. We show how to practically model this formalism by using a new conditional U-Net architecture, which fully takes into account the conditional information and does not need any fixed image. Our formalism is then applied to image moving tumors for radiotherapy treatment at different breathing amplitude using 4D-CT (3D+t) scans in thoracoabdominal regions. This driving application is particularly complex as it requires to stitch a collection of sequential 2D slices into several 3D volumes at different organ positions. Movement interpolation with standard methods then generates well known reconstruction artefacts in the assembled volumes due to irregular patient breathing, hysteresis and poor correlation of breathing signal to internal motion. Results obtained on 4D-CT clinical data showcase artefact-free volumes achieved through real-time latencies. The code is publicly available at https://github.com/Kheil-Z/IMITATE .

Figures (2)

• Figure 1: (left) Our method recovers the image motion at phase amplitudes $a_{\varphi(t)}$ using images acquired at different phases $M_i$, $i \in \{1,\cdots,n\}$. Classic approaches only use the two images, denoted $(M,F)$, acquired just before and after phase $\varphi(t)$. (right) Example of recovered breathing signals for a slice at amplitude $a_{\varphi}$: (A) Default reconstructions, uses image M, (B) using $(M,F)$ with interpolated deformation, and (C) using the proposed method with $M_i$, $i \in \{1,\cdots,n\}$.
• Figure 2: Main framework pipeline. F is the unseen, reference image, $M_1,...M_n$ the moving contextual frames and their associated signals $\delta_i$. Additional segmentation labels can be used like in balakrishnan_voxelmorph_2019 (Sec \ref{['sec:ref_agno_reg_frame']}). Figure also displays a convolutional block from the proposed Conditional U-Net. $\mathcal{F}_{n-1}$ is the feature map at depth n, $FC_n$ the fully connected layer which produces $(s_1,s_2)$, and $Conv^{1}_n$,$Conv^{2}_n$ the convolution layers. See section \ref{['sec:model']}