A Hybrid Deep Feature-Based Deformable Image Registration Method for Pathology Images

TL;DR

A hybrid deep feature-based deformable image registration framework for stained pathology samples is proposed and can potentially become a reliable method for pathology image registration.

Abstract

Pathologists need to combine information from differently stained pathology slices for accurate diagnosis. Deformable image registration is a necessary technique for fusing multi-modal pathology slices. This paper proposes a hybrid deep feature-based deformable image registration framework for stained pathology samples. We first extract dense feature points via the detector-based and detector-free deep learning feature networks and perform points matching. Then, to further reduce false matches, an outlier detection method combining the isolation forest statistical model and the local affine correction model is proposed. Finally, the interpolation method generates the deformable vector field for pathology image registration based on the above matching points. We evaluate our method on the dataset of the Non-rigid Histology Image Registration (ANHIR) challenge, which is co-organized with the IEEE ISBI 2019 conference. Our technique outperforms the traditional approaches by 17% with the Average-Average registration target error (rTRE) reaching 0.0034. The proposed method achieved state-of-the-art performance and ranked 1st in evaluating the test dataset. The proposed hybrid deep feature-based registration method can potentially become a reliable method for pathology image registration.

Figures (12)

• Figure 1: Examples of pathology images of each site.
• Figure 2: A visual comparison of the results of our method with those of traditional manual algorithms.
• Figure 3: Workflow of the proposed method. Generate Feature Point Paris: We extract multiple pairs of feature points from different networks. Refine match pairs and get deformable fields: The point pairs are combined and passed into the outlier detection module. Finally, new point pairs are obtained, and the DVF is generated by interpolation. Fine-Tune: First, we employ a pre-trained model in natural images on our inference. Then, we use the inference output to construct pseudo-labels and landmarks to train the network together. The final trained model is obtained to achieve more accurate results.
• Figure 4: The Detector-Free Matching Network comprise the feature extraction and Transformer module. After approaching the CNN network to extract features, the two images go through an encoding-decoding-decoding procedure to get the coordinates x in the fixed image corresponding to ${x}^{'}$ in the moving image.
• Figure 5: Two parts of detector-based matching network. The first part is the detector network (a), a self-supervised feature point extraction and descriptor construction network. The second part is the matching network (b), a feature matching network based on graph networks.