From Traditional to Deep Learning Approaches in Whole Slide Image Registration: A Methodological Review

TL;DR

This methodological review tackles the challenge of registering gigapixel whole-slide histopathology images by surveying conventional and deep learning approaches, datasets, evaluation metrics, and software. It formulates registration as an optimization over transformations that minimize a similarity cost between moving and reference WSIs, and emphasizes the unique difficulties posed by multi-stain, multi-slide, and artefact-prone data. The paper catalogs conventional methods (intensity, feature, frequency-domain, and point-set) and three deep-learning categories (feature extraction, parameter estimation, and generative models), highlighting their strengths, limitations, and representative studies. It also assesses publicly available datasets (e.g., ANHIR, ACROBAT, HyReCo) and software tools, discusses evaluation strategies (TRE, Dice, MI, etc.), and outlines open challenges and future directions, such as robust artefact handling and multi-scale, multi-slide integration. Overall, while DL approaches offer substantial speed and automation advantages, they have yet to consistently outperform traditional deformable registration, signaling a fertile ground for future advances and practical, generalizable WSI registration solutions.

Abstract

Whole slide image (WSI) registration is an essential task for analysing the tumour microenvironment (TME) in histopathology. It involves the alignment of spatial information between WSIs of the same section or serial sections of a tissue sample. The tissue sections are usually stained with single or multiple biomarkers before imaging, and the goal is to identify neighbouring nuclei along the Z-axis for creating a 3D image or identifying subclasses of cells in the TME. This task is considerably more challenging compared to radiology image registration, such as magnetic resonance imaging or computed tomography, due to various factors. These include gigapixel size of images, variations in appearance between differently stained tissues, changes in structure and morphology between non-consecutive sections, and the presence of artefacts, tears, and deformations. Currently, there is a noticeable gap in the literature regarding a review of the current approaches and their limitations, as well as the challenges and opportunities they present. We aim to provide a comprehensive understanding of the available approaches and their application for various purposes. Furthermore, we investigate current deep learning methods used for WSI registration, emphasising their diverse methodologies. We examine the available datasets and explore tools and software employed in the field. Finally, we identify open challenges and potential future trends in this area of research.

Figures (7)

• Figure 1: Overview of a general pipeline for WSI registration and downstream analysis. After biopsy collection and digitization, the registration algorithm matches two images. The resulting registered images are then used for further analysis. In this figure, the downstream analysis is adopted from the work of Westein et al. wetstein2022deep, which focuses on the analysis of survival rates.
• Figure 2: Breakdown of papers proposed for histology image registration (included in this review) in the year of publication. The chart depicts rising trends in the utilisation of deep learning for WSI registration.
• Figure 3: Taxonomy of registration methods reviewed in this study. The methods are categorised based on their approach into two main groups: deep learning models and conventional approaches. General models used for each category and reviewed in this work are depicted in pink.
• Figure 4: A: An example of WSI with a magnified view of the tumour region. Both views depict the same tissue sample: the left stained with HE for general tissue structure, and the right stained with IHC to highlight specific protein expression qaiser2018her. Each image has a resolution of approximately 100K $\times$ 50K pixels. B: Comparison of traditional medical image and WSI registration. The top row displays the registration of CT scan and MRI images. From left to right: CT scan, MRI, and the resulting overlap of registration gui2023normal. The bottom row depicts the registration of patches from IHC and HE images. From left to right: IHC image, HE image, and the registration overlapping results displayed as a false-colour image trahearn2017hyper. In radiology image registration, both images often share common baselines that help in alignment, whereas histopathology images typically lack such features, making registration more challenging.
• Figure 5: Example images depicting challenges for WSI registration due to imperfections in the digitisation process. These artefacts have been sampled from the ACROBAT 2023 challenge dataset weitz_acrobat_2023.