Structure-Guided MR-to-CT Synthesis with Spatial and Semantic Alignments for Attenuation Correction of Whole-Body PET/MR Imaging

TL;DR

A novel whole-body MR-to-CT synthesis framework, which consists of three novel modules to tackle several challenges including the spatial misalignment caused by tissue variety and respiratory movements, and the complex intensity mapping due to large intensity variations across the whole body is proposed.

Abstract

Deep-learning-based MR-to-CT synthesis can estimate the electron density of tissues, thereby facilitating PET attenuation correction in whole-body PET/MR imaging. However, whole-body MR-to-CT synthesis faces several challenges including the issue of spatial misalignment and the complexity of intensity mapping, primarily due to the variety of tissues and organs throughout the whole body. Here we propose a novel whole-body MR-to-CT synthesis framework, which consists of three novel modules to tackle these challenges: (1) Structure-Guided Synthesis module leverages structure-guided attention gates to enhance synthetic image quality by diminishing unnecessary contours of soft tissues; (2) Spatial Alignment module yields precise registration between paired MR and CT images by taking into account the impacts of tissue volumes and respiratory movements, thus providing well-aligned ground-truth CT images during training; (3) Semantic Alignment module utilizes contrastive learning to constrain organ-related semantic information, thereby ensuring the semantic authenticity of synthetic CT images.We conduct extensive experiments to demonstrate that the proposed whole-body MR-to-CT framework can produce visually plausible and semantically realistic CT images, and validate its utility in PET attenuation correction.

Figures (6)

• Figure 1: Overview of the proposed whole-body MR-to-CT synthesis framework. (a) Structure-Guided Synthesis (SGSyn) translates the input WFI-IP & WFI-OP MR images $I_{in}^{MR}$ into a synthetic CT image $I_{syn}^{CT}$, which is the main branch of our framework. (b) Spatial Alignment (SpatAlign) registers $I_{in}^{MR}$ and the paired CT scan $I_{in}^{CT}$ via a registration network $R(\cdot)$, resulting in the aligned CT image $I_{align}^{CT}$ that provides an explicit similarity constraint on $I_{syn}^{CT}$. (c) Semantic Alignment (SemAlign) utilizes contrastive learning to enforce the similarity of identical organs and the dissimilarity of different organs between the feature representation $S_{syn}$ and $S_{align}$, thus ensuring the semantic authenticity of the synthetic results.
• Figure 2: The architecture of the synthesis network from Structure-Guided Synthesis (SGSyn). The network consists of one shared image encoder and two decoders to output a synthetic CT image and its edge map, respectively. It leverages structure-guided attention gates at different scales to facilitate information flow between two decoders for enhancing synthetic CT image quality.
• Figure 3: Qualitative comparison between the proposed method and other MR-to-CT synthesis models. The columns from left to right include (a) the input WFI-IP MR image, (b) the input WFI-OP MR image, (c) the reference CT image (not well-aligned with MR images), and the synthetic CT images using (d) our method, (e) gc-CycleGAN hiasa2018cross, (f) sc-CycleGAN yang2020unsupervised, and (h) CycleGAN zhu2017unpaired, respectively.
• Figure 4: Qualitative results from ablation study of key designs in three modules: (1) the gated U-Net design in SGSyn; (2) the tissue-aware image similarity constraint and the respiration-aware smoothness regularization in SpatAlign; (3) the contrastive loss in SemAlign.
• Figure 5: The respiration-aware smoothness regularization $\mathcal{L}_{smooth}$ enforces a deformation field that is less smooth near the thoracic cavity contour, thus preventing the ribs from sliding. It can be observed along the yellow line that although abdominal organs are precisely registered, there is a noticeable slippage of the ribs when $\mathcal{L}_{smooth}$ is not used.