Unsupervised Deformable Image Registration with Local-Global Attention and Image Decomposition

TL;DR

This work addresses unsupervised deformable image registration in medical imaging, particularly under large displacements and cross-modality scenarios. It introduces LGANet++, a pyramid-shaped framework built on a dual-stream encoder, local-global attention (LGAM), and a feature interaction/fusion pipeline (FIFM) with a multi-scale fusion module (MSFM) to iteratively refine deformation fields via four stages $\phi_4,\phi_3,\phi_2,\phi_1$, while enforcing diffeomorphic, smooth transformations through a differentiable transform layer. The approach achieves state-of-the-art performance across five public datasets in cross-patient, cross-time, and cross-modal registration, with quantitative gains (e.g., up to $6.12\%$ in cross-modal CT-MR) and robust generalization to domain shifts. The method relies on a $\mathcal{L}$-style loss combining local NCC similarity with a gradient-based regularizer, enabling fast, unsupervised inference suitable for clinical workflows, and it demonstrates strong potential for intraoperative navigation and longitudinal imaging analysis in multi-modal contexts.

Abstract

Deformable image registration is a critical technology in medical image analysis, with broad applications in clinical practice such as disease diagnosis, multi-modal fusion, and surgical navigation. Traditional methods often rely on iterative optimization, which is computationally intensive and lacks generalizability. Recent advances in deep learning have introduced attention-based mechanisms that improve feature alignment, yet accurately registering regions with high anatomical variability remains challenging. In this study, we proposed a novel unsupervised deformable image registration framework, LGANet++, which employs a novel local-global attention mechanism integrated with a unique technique for feature interaction and fusion to enhance registration accuracy, robustness, and generalizability. We evaluated our approach using five publicly available datasets, representing three distinct registration scenarios: cross-patient, cross-time, and cross-modal CT-MR registration. The results demonstrated that our approach consistently outperforms several state-of-the-art registration methods, improving registration accuracy by 1.39% in cross-patient registration, 0.71% in cross-time registration, and 6.12% in cross-modal CT-MR registration tasks. These results underscore the potential of LGANet++ to support clinical workflows requiring reliable and efficient image registration. The source code is available at https://github.com/huangzyong/LGANet-Registration.

Figures (9)

• Figure 1: Illustration of our proposed LGANet++. Two structurally identical encoders (sharing weights) extract features from the moving image and the reference image, respectively. A multi-scale fusion module then combines these two feature sets. Finally, the decoder performs deformation field ($\phi_i$) estimation based on the fused features ($C_i$), moving image features ($M_i$), and reference image features ($F_i$). The deformation fields are progressively optimized from $\phi_4$ to $\phi_1$ in a coarse-to-fine manner, with $\phi_1$ representing the final deformation field. MSFM: multi-scale fusion module, which is used to fuse moving image features and fixed image features. LGAM: local-global attention module, which is used to estimate the initial deformation field $\phi_4$. FIFM: feature interaction and fusion module, which is used to estimate and refine the deformation field at each stage of the decoding process. Diff: diffeomorphic layer, which ensures smooth, invertible, and topology-preserving transformations by integrating a differentiable exponential mapping into the deformation field prediction.
• Figure 2: Depiction of the proposed local and global attention module (LGAM) and the feature interaction and fusion module (FIFM). (a) Computational procedure of LGAM. Features from different branches are concatenated and then processed through a positional attention module (PAM) to capture spatial dependencies. This is followed by attention mechanisms operating at both global and local levels to extract integrated and fine-grained features. (b) Feature interaction and fusion module (FIFM). $Corr$ indicates a 3D correlation layer that computes pixel-wise similarity relationships between the two feature maps. (c) Image decomposition module (IDM).
• Figure 3: (a) Architecture of the Position Attention Module (PAM). The input features are processed through convolutional layers (blue arrows) to generate Q, K, V representations. The self-attention mechanism is applied after reshaping, followed by a Softmax operation ($\textcircled{s}$) to compute spatial dependencies. The output is then reconstructed and enhanced via a residual block to produce refined features $U$. (b) Structure of the Channel-wise Attention Module (CWAM). The top part illustrates the adaptive channel fusion block, which learns channel-wise weights for feature maps $C_i$, $I_w$, and $I_f$ through a multi-channel attention (MCA) mechanism. The bottom part consists of a SENet-based squeeze-excitation block, a ResNet block, and a convolutional layer, collectively used to selectively emphasize informative channels and output the final deformation field $\phi$.
• Figure 4: Illustration of the optimization strategy for refining the deformation field in a coarse-to-fine manner. Here, $\phi_{i}'$ denotes the initial output deformation field in the current stage. $\phi_{i}$ signifies the deformation field acquired subsequent to the optimization process using the previous stage $\phi_{i+1}$, which possesses a lower resolution and captures relatively coarser deformation characteristics.
• Figure 5: Comparing the visual results obtained by different methods on the LPBA dataset. The third row is the residual maps between warped moving images and fixed images ($I_{residual}=I_f-I_w$), with the mean absolute error ($error=mean(|I_{residual}|)$) placed in the bottom right corner. A cleaner error map indicates a better registration result.