Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Co-Learning Semantic-aware Unsupervised Segmentation for Pathological Image Registration

Yang Liu, Shi Gu

TL;DR

This work tackles pathology image registration where focal lesions disrupt spatial correspondence. It introduces GIRNet, a fully unsupervised tri-network framework (RegNet, SegNet, InpNet) that co-learns registration, segmentation, and inpainting to explicitly address non-correspondent regions through semantic guidance and mutual information minimization. Key contributions include a collaborative optimization strategy, an unsupervised segmentation approach based on minimal mutual information, and demonstrated gains in atlas-based and longitudinal brain MRI tasks with competitive lesion identification. The approach offers a cost-effective, annotation-free pathway to robust pathological image registration, with code released for reproducibility and further research.

Abstract

The registration of pathological images plays an important role in medical applications. Despite its significance, most researchers in this field primarily focus on the registration of normal tissue into normal tissue. The negative impact of focal tissue, such as the loss of spatial correspondence information and the abnormal distortion of tissue, are rarely considered. In this paper, we propose GIRNet, a novel unsupervised approach for pathological image registration by incorporating segmentation and inpainting through the principles of Generation, Inpainting, and Registration (GIR). The registration, segmentation, and inpainting modules are trained simultaneously in a co-learning manner so that the segmentation of the focal area and the registration of inpainted pairs can improve collaboratively. Overall, the registration of pathological images is achieved in a completely unsupervised learning framework. Experimental results on multiple datasets, including Magnetic Resonance Imaging (MRI) of T1 sequences, demonstrate the efficacy of our proposed method. Our results show that our method can accurately achieve the registration of pathological images and identify lesions even in challenging imaging modalities. Our unsupervised approach offers a promising solution for the efficient and cost-effective registration of pathological images. Our code is available at https://github.com/brain-intelligence-lab/GIRNet.

Co-Learning Semantic-aware Unsupervised Segmentation for Pathological Image Registration

TL;DR

This work tackles pathology image registration where focal lesions disrupt spatial correspondence. It introduces GIRNet, a fully unsupervised tri-network framework (RegNet, SegNet, InpNet) that co-learns registration, segmentation, and inpainting to explicitly address non-correspondent regions through semantic guidance and mutual information minimization. Key contributions include a collaborative optimization strategy, an unsupervised segmentation approach based on minimal mutual information, and demonstrated gains in atlas-based and longitudinal brain MRI tasks with competitive lesion identification. The approach offers a cost-effective, annotation-free pathway to robust pathological image registration, with code released for reproducibility and further research.

Abstract

The registration of pathological images plays an important role in medical applications. Despite its significance, most researchers in this field primarily focus on the registration of normal tissue into normal tissue. The negative impact of focal tissue, such as the loss of spatial correspondence information and the abnormal distortion of tissue, are rarely considered. In this paper, we propose GIRNet, a novel unsupervised approach for pathological image registration by incorporating segmentation and inpainting through the principles of Generation, Inpainting, and Registration (GIR). The registration, segmentation, and inpainting modules are trained simultaneously in a co-learning manner so that the segmentation of the focal area and the registration of inpainted pairs can improve collaboratively. Overall, the registration of pathological images is achieved in a completely unsupervised learning framework. Experimental results on multiple datasets, including Magnetic Resonance Imaging (MRI) of T1 sequences, demonstrate the efficacy of our proposed method. Our results show that our method can accurately achieve the registration of pathological images and identify lesions even in challenging imaging modalities. Our unsupervised approach offers a promising solution for the efficient and cost-effective registration of pathological images. Our code is available at https://github.com/brain-intelligence-lab/GIRNet.
Paper Structure (17 sections, 9 equations, 7 figures, 1 table)

This paper contains 17 sections, 9 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Collaborative optimization
  4. Network Modules
  5. RegNet.
  6. SegNet.
  7. InpNet.
  8. Experiments
  9. Dataset and Pre-processing
  10. Comparison to pathology registration
  11. Unsupervised segmentation
  12. Ablation study
  13. Conclusion
  14. Acknowledgements
  15. Details on Minimal Mutual Information
  16. ...and 2 more sections

Figures (7)

  • Figure 1: The proposed tri-modules collaborative learning framework for medical image analysis includes RegNet, SegNet, and InpNet to achieve accurate image registration and segmentation through the optimization of semantic-informed mutual information.
  • Figure 2: Boxplots of mean deformation errors with respect to the gold standard deformations in three different regions on the pseudo dataset. Left to right: in tumor, near tumor and far from tumor.
  • Figure 3: Boxplots of the average target registration error (TRE) in two different regions: near tumor (left) and far from tumor (right).
  • Figure 4: Registration and segmentation results for Pseudo dataset. The 7 columns show: 1) the moving image; 2) the atlas; 3) the inpainted image; 4) the warped inpainted image; 5) the warped atlas image; 6) the ground truth mask 7) the predicted mask.
  • Figure 5: Example of MR slices from a pathological brain image, which is an undesirable segmentation of brain tissue and black background based on mutual information theory without semantic information provided by RegNet.
  • ...and 2 more figures