RegWSI: Whole Slide Image Registration using Combined Deep Feature- and Intensity-Based Methods: Winner of the ACROBAT 2023 Challenge

TL;DR

RegWSI presents a robust, dataset-agnostic WSI registration framework that combines a deep-feature-based initial alignment with an intensity-based multilevel instance-optimization for nonrigid refinement. The method uses a multi-scale, multi-angle initial alignment based on SuperPoint and SuperGlue, followed by a local NCC-based deformable registration with diffusive regularization, formalized as $O_{REG}(S,T,u)=NCC(S_i\circ u_i,T_i)+\theta_i Reg(u_i)$. It achieves first place on ACROBAT 2023 and competitive performance on ANHIR and HyReCo, without dataset-specific fine-tuning, and is released within the DeeperHistReg framework for out-of-the-box use on diverse microscopic images. This work advances practical digital pathology by enabling accurate cross-stain WSI fusion and efficient annotation transfer across slides, reducing time and costs in multi-stain diagnostics and research.

Abstract

The automatic registration of differently stained whole slide images (WSIs) is crucial for improving diagnosis and prognosis by fusing complementary information emerging from different visible structures. It is also useful to quickly transfer annotations between consecutive or restained slides, thus significantly reducing the annotation time and associated costs. Nevertheless, the slide preparation is different for each stain and the tissue undergoes complex and large deformations. Therefore, a robust, efficient, and accurate registration method is highly desired by the scientific community and hospitals specializing in digital pathology. We propose a two-step hybrid method consisting of (i) deep learning- and feature-based initial alignment algorithm, and (ii) intensity-based nonrigid registration using the instance optimization. The proposed method does not require any fine-tuning to a particular dataset and can be used directly for any desired tissue type and stain. The method scored 1st place in the ACROBAT 2023 challenge. We evaluated using three open datasets: (i) ANHIR, (ii) ACROBAT, and (iii) HyReCo, and performed several ablation studies concerning the resolution used for registration and the initial alignment robustness and stability. The method achieves the most accurate results for the ACROBAT dataset, the cell-level registration accuracy for the restained slides from the HyReCo dataset, and is among the best methods evaluated on the ANHIR dataset. The method does not require any fine-tuning to a new datasets and can be used out-of-the-box for other types of microscopic images. The method is incorporated into the DeeperHistReg framework, allowing others to directly use it to register, transform, and save the WSIs at any desired pyramid level. The proposed method is a significant contribution to the WSI registration, thus advancing the field of digital pathology.

Figures (9)

• Figure 1: Exemplary registration pairs from the ACROBAT, ANHIR, and HyReCo datasets. Note the clinical quality of the ACROBAT samples (without any initial preprocessing and numerous artifacts), large initial misalignment in the ANHIR dataset, and the good quality of the HyReCo samples.
• Figure 2: The pipeline of the proposed method. The processing starts with the preprocessing consisting of resampling and color normalization, followed by the robust initial alignment, and finally the accurate deformable registration. The final warping is performed in a separate module at the original resolution.
• Figure 3: The multi-scale and multi-angle initial alignment pipeline. Since the keypoint extractors are never perfectly scale- and rotation-invariant, such an exhaustive procedure improves the alignment robustness. The transform with the largest number of matches is chosen as the final one, choosing from all transformations calculated across the predefined scales and rotation angles. The procedure improves the alignment quality especially for outliers which would be otherwise incorrectly registered, potentially even decreasing the results quality with respect to the initial misalignment.
• Figure 4: Visual registration results for exemplary cases from all three datasets. The results are presented on a chosen ROI after subsequent registration steps. The global overview for ANHIR case is rotated, and the ACROBAT source image is cropped, for the presentation clarity. Note the difference between the registration quality of restained (HyReCo) and consecutive slides (ACROBAT/ANHIR).
• Figure 5: The quantitative results in terms of the target registration error (TRE) after the subsequent registration steps. The HyReCo results are reported for the consecutive slides only. Note the significant improvement (p-value < 0.01) after each registration step.