FIRE: Unsupervised bi-directional inter-modality registration using deep networks

TL;DR

This work tackles unsupervised inter-modality image registration by learning bidirectional affine and non-rigid transformations with inverse-consistency constraints. The proposed FIRE framework employs a modality-independent synthesis encoder $G$, two decoders $F^{A\rightarrow B}$ and $F^{B\rightarrow A}$, and dual transformation networks $T^{A\rightarrow B}$ and $T^{B\rightarrow A}$ to perform cross-modality synthesis and warping, optimized via the composite loss $\mathcal{L} = \mathcal{L}_{syn} + \mathcal{L}_{reg} + \mathcal{R}$. Experiments on multi-sequence MRBrainS data and intra-modality 4D cardiac cine-MR (ACDC) show FIRE achieves state-of-the-art or competitive Dice scores, notably resolving difficult IR–FLAIR alignment where baselines struggle. The approach provides a robust, unsupervised, end-to-end framework for pre-processing in clinical pipelines, with potential to improve cross-modality fusion and downstream analyses.

Abstract

Inter-modality image registration is an critical preprocessing step for many applications within the routine clinical pathway. This paper presents an unsupervised deep inter-modality registration network that can learn the optimal affine and non-rigid transformations simultaneously. Inverse-consistency is an important property commonly ignored in recent deep learning based inter-modality registration algorithms. We address this issue through the proposed multi-task architecture and the new comprehensive transformation network. Specifically, the proposed model learns a modality-independent latent representation to perform cycle-consistent cross-modality synthesis, and use an inverse-consistent loss to learn a pair of transformations to align the synthesized image with the target. We name this proposed framework as FIRE due to the shape of its structure. Our method shows comparable and better performances with the popular baseline method in experiments on multi-sequence brain MR data and intra-modality 4D cardiac Cine-MR data.

Figures (6)

• Figure 1: Architecture of the FIRE model: a synthesis encoder, $G$, that extracts modality-independent features; two synthesis decoders, $F^{A\rightarrow B}$ and $F^{B\rightarrow A}$, that map the features extracted by $G$ to synthesized images; and two transformation networks, $T^{A\rightarrow B}$ and $T^{B\rightarrow A}$, that predict the transformation fields.
• Figure 2: Architecture of the synthesis encoder and decoders.
• Figure 3: Architecture of the transformation networks.
• Figure 4: Representative results of MRBrainS T1-FLAIR data.The outer contour of cerebrospinal fluid in the extracerebral space segmented on T1 images are shown in blue.
• Figure 5: Example results of registering the IR