Path and Bone-Contour Regularized Unpaired MRI-to-CT Translation

TL;DR

This work tackles unpaired MRI-to-CT translation with a focus on preserving bone contours for radiotherapy. It introduces a latent-space transfer modeled as a neural ordinary differential equation, with path-length regularization to select the optimal flow. It adds a bone-contour regularization using a trainable contour generator and an attention mechanism to emphasize bone-adjacent regions. Experiments on head, pelvic, and SynthRAD brain datasets show superior image fidelity (PSNR/SSIM/LPIPS) and improved bone segmentation Dice (~0.84) compared to baselines, and the method offers faster inference than diffusion models. The approach has practical potential for improving radiotherapy dose planning and can generalize to other anatomically significant structures; code is publicly available.

Abstract

Accurate MRI-to-CT translation promises the integration of complementary imaging information without the need for additional imaging sessions. Given the practical challenges associated with acquiring paired MRI and CT scans, the development of robust methods capable of leveraging unpaired datasets is essential for advancing the MRI-to-CT translation. Current unpaired MRI-to-CT translation methods, which predominantly rely on cycle consistency and contrastive learning frameworks, frequently encounter challenges in accurately translating anatomical features that are highly discernible on CT but less distinguishable on MRI, such as bone structures. This limitation renders these approaches less suitable for applications in radiation therapy, where precise bone representation is essential for accurate treatment planning. To address this challenge, we propose a path- and bone-contour regularized approach for unpaired MRI-to-CT translation. In our method, MRI and CT images are projected to a shared latent space, where the MRI-to-CT mapping is modeled as a continuous flow governed by neural ordinary differential equations. The optimal mapping is obtained by minimizing the transition path length of the flow. To enhance the accuracy of translated bone structures, we introduce a trainable neural network to generate bone contours from MRI and implement mechanisms to directly and indirectly encourage the model to focus on bone contours and their adjacent regions. Evaluations conducted on three datasets demonstrate that our method outperforms existing unpaired MRI-to-CT translation approaches, achieving lower overall error rates. Moreover, in a downstream bone segmentation task, our approach exhibits superior performance in preserving the fidelity of bone structures. Our code is available at: https://github.com/kennysyp/PaBoT.

Figures (7)

• Figure 1: Qualitative comparison of the proposed approach with mainstream unpaired MRI-to-CT translation methods. The top and bottom rows depict zoomed-out regions from the head and the pelvic datasets, respectively. Our method demonstrates superior bone fidelity compared to competing approaches.
• Figure 2: A schematic overview of Cycle Consistency, Contrastive Learning, and the proposed framework. Here, $x$ and $y$ represent the MRI and CT images, respectively. (1) $y_{G_1}$ is the synthetic CT, while $x_{G_2}$ is the synthetic MRI to ensure cycle consistency. $G_1$ and $G_2$ denote respective generators. (2) $E$ and $D$ denote the encoder and decoder. The latent representation of the CT image is given by $\hat{y}$, and the contrastive loss is denoted by $\mathit{L}_{\mathit{patch}\mathrm{NCE}}$. (3) $\hat{x}$ and $\hat{y}$ correspond to the latent representation of MRI and CT images. $E$ and $G$ refer to the encoder and decoder modules, and $\theta\in[0,1]$ is a continuous time variable.
• Figure 3: Illustration of the intuition of the path regularization. Translating an MRI image $x_1$ to its true CT counterpart $y_1$ should require fewer cumulative changes and less transformation effort than mapping it to an alternative CT point $y_2$, resulting in a shorter transition path length.
• Figure 4: A more detailed illustration of the proposed framework, with particular emphasis on the layer-wise path-length regularization and bone-contour regularization components.
• Figure 5: Qualitative comparison for all translated CTs. From left to right, the source MRI, the reference CT, and all translated CTs. From top to bottom, one example is presented for each of the Head, Pelvic, and SynthRAD datasets.