UTSRMorph: A Unified Transformer and Superresolution Network for Unsupervised Medical Image Registration

TL;DR

A novel unsupervised image registration method named the unified Transformer and superresolution (UTSRMorph) network, which can enhance feature representation learning in the encoder and generate detailed displacement fields in the decoder to overcome problems of feature degradation.

Abstract

Complicated image registration is a key issue in medical image analysis, and deep learning-based methods have achieved better results than traditional methods. The methods include ConvNet-based and Transformer-based methods. Although ConvNets can effectively utilize local information to reduce redundancy via small neighborhood convolution, the limited receptive field results in the inability to capture global dependencies. Transformers can establish long-distance dependencies via a self-attention mechanism; however, the intense calculation of the relationships among all tokens leads to high redundancy. We propose a novel unsupervised image registration method named the unified Transformer and superresolution (UTSRMorph) network, which can enhance feature representation learning in the encoder and generate detailed displacement fields in the decoder to overcome these problems. We first propose a fusion attention block to integrate the advantages of ConvNets and Transformers, which inserts a ConvNet-based channel attention module into a multihead self-attention module. The overlapping attention block, a novel cross-attention method, uses overlapping windows to obtain abundant correlations with match information of a pair of images. Then, the blocks are flexibly stacked into a new powerful encoder. The decoder generation process of a high-resolution deformation displacement field from low-resolution features is considered as a superresolution process. Specifically, the superresolution module was employed to replace interpolation upsampling, which can overcome feature degradation. UTSRMorph was compared to state-of-the-art registration methods in the 3D brain MR (OASIS, IXI) and MR-CT datasets. The qualitative and quantitative results indicate that UTSRMorph achieves relatively better performance. The code and datasets are publicly available at https://github.com/Runshi-Zhang/UTSRMorph.

Figures (10)

• Figure 1: The unified Transformer and superresolution (UTSRMorph) network include four stages in the encoder and several SR modules in the decoder. One stage consists of FABs and an OAB. The black dashed arrows represent that the moving image and fixed image are concatenated as input to the following two optional convolutional blocks.
• Figure 2: Overlapping window partition of OAB.
• Figure 3: Schematic diagram of the pixel-shuffle layer in SR module.
• Figure 4: Boxplots with Dice scores of various registration methods in the IXI dataset.
• Figure 5: The visualization results of various registration methods in the IXI dataset. The results include the raw image and ground-truth mask of the moving and fixed images, warped image and warped mask, estimated continuous and RGB deformations.The RGB deformation is generated by mapping each spatial dimension of the displacement field to a channel of the RGB color.