Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nuclei-Location Based Point Set Registration of Multi-Stained Whole Slide Images

Adith Jeyasangar, Abdullah Alsalemi, Shan E Ahmed Raza

TL;DR

The paper addresses the challenge of registering multi-stained whole slide images at the nuclei level to enable downstream tumor microenvironment analyses. It proposes a nuclei-location based point-set registration pipeline that detects nuclei with Hover-Net, performs rigid alignment via Automatic Rotation Alignment and Iterative Closest Point, and refines using a non-rigid, LLE-guided CPD-like method followed by TPS and B-spline warping. Evaluated on HYRECO subset B with H&E and PHH3 stains, the method achieves lower nucleus-level registration error than the DFBR baseline, with an average rTRE around 1.63×10^-2 and further improvements to about 1.28×10^-2 after B-spline refinement. The approach is generalizable to other stains given a robust nuclei detector and offers a practical, faster alternative to deep-learning-only registrations for precise nuclei-level alignment, enabling more reliable downstream analyses of cell subtypes and biomarker signatures in the tumor microenvironment.

Abstract

Whole Slide Images (WSIs) provide exceptional detail for studying tissue architecture at the cell level. To study tumour microenvironment (TME) with the context of various protein biomarkers and cell sub-types, analysis and registration of features using multi-stained WSIs is often required. Multi-stained WSI pairs normally suffer from rigid and non-rigid deformities in addition to slide artefacts and control tissue which present challenges at precise registration. Traditional registration methods mainly focus on global rigid/non-rigid registration but struggle with aligning slides with complex tissue deformations at the nuclei level. However, nuclei level non-rigid registration is essential for downstream tasks such as cell sub-type analysis in the context of protein biomarker signatures. This paper focuses on local level non-rigid registration using a nuclei-location based point set registration approach for aligning multi-stained WSIs. We exploit the spatial distribution of nuclei that is prominent and consistent (to a large level) across different stains to establish a spatial correspondence. We evaluate our approach using the HYRECO dataset consisting of 54 re-stained images of H\&E and PHH3 image pairs. The approach can be extended to other IHC and IF stained WSIs considering a good nuclei detection algorithm is accessible. The performance of the model is tested against established registration algorithms and is shown to outperform the model for nuclei level registration.

Nuclei-Location Based Point Set Registration of Multi-Stained Whole Slide Images

TL;DR

The paper addresses the challenge of registering multi-stained whole slide images at the nuclei level to enable downstream tumor microenvironment analyses. It proposes a nuclei-location based point-set registration pipeline that detects nuclei with Hover-Net, performs rigid alignment via Automatic Rotation Alignment and Iterative Closest Point, and refines using a non-rigid, LLE-guided CPD-like method followed by TPS and B-spline warping. Evaluated on HYRECO subset B with H&E and PHH3 stains, the method achieves lower nucleus-level registration error than the DFBR baseline, with an average rTRE around 1.63×10^-2 and further improvements to about 1.28×10^-2 after B-spline refinement. The approach is generalizable to other stains given a robust nuclei detector and offers a practical, faster alternative to deep-learning-only registrations for precise nuclei-level alignment, enabling more reliable downstream analyses of cell subtypes and biomarker signatures in the tumor microenvironment.

Abstract

Whole Slide Images (WSIs) provide exceptional detail for studying tissue architecture at the cell level. To study tumour microenvironment (TME) with the context of various protein biomarkers and cell sub-types, analysis and registration of features using multi-stained WSIs is often required. Multi-stained WSI pairs normally suffer from rigid and non-rigid deformities in addition to slide artefacts and control tissue which present challenges at precise registration. Traditional registration methods mainly focus on global rigid/non-rigid registration but struggle with aligning slides with complex tissue deformations at the nuclei level. However, nuclei level non-rigid registration is essential for downstream tasks such as cell sub-type analysis in the context of protein biomarker signatures. This paper focuses on local level non-rigid registration using a nuclei-location based point set registration approach for aligning multi-stained WSIs. We exploit the spatial distribution of nuclei that is prominent and consistent (to a large level) across different stains to establish a spatial correspondence. We evaluate our approach using the HYRECO dataset consisting of 54 re-stained images of H\&E and PHH3 image pairs. The approach can be extended to other IHC and IF stained WSIs considering a good nuclei detection algorithm is accessible. The performance of the model is tested against established registration algorithms and is shown to outperform the model for nuclei level registration.
Paper Structure (16 sections, 3 equations, 5 figures, 1 table)

This paper contains 16 sections, 3 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Dataset
  5. Evaluation Metrics
  6. Overview of Registration Pipeline
  7. Nuclei Detection
  8. Rigid Registration
  9. Automatic Rotation Alignment (ARA)
  10. Iterative Closest Point (ICP)
  11. Non-Rigid Registration
  12. Point Set Registration
  13. Image Warping
  14. B-Spline Non-Rigid Registration
  15. Results and Discussion
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Tiles extracted from H&E and PHH3 slides. Row a) Shows H&E slides with green cross indicating nuclei. Row b) Shows PHH3 slides with red cross indicating nuclei.
  • Figure 2: Overview of nuclei-location based point set registration pipeline. First, the WSI pair is roughly aligned by a rigid registration step. Second, image tiles are extracted from both images where a fixed and moving tile undergoes a pre-processing stage followed by nuclei detection. Following, the point-set registration model carries out pre-alignment, ICP, and non-rigid point-set registration operations. Once a transformation matrix is obtained, it is applied to the moving image, producing a wrapped image. A further refined non-rigid registration step using the B-Spline method is conducted to obtained the final result.
  • Figure 3: Overlay of a sample fixed and warped image tiles using the proposed pipeline. The red circles identifies the mis-alignments that can be fixed by using B-Spline non rigid registration.
  • Figure 4: Sample tile of a (a) fixed Image, (b) warped image using DFBR, (c) warped image from proposed method, and (d) warped image using proposed method + B-spline, followed by an overlay of fixed image and warped images (e) with original moving image, (f) with warped image using DFBR, (g) with warped image using the proposed method, and (h) with warped image using the proposed method + B-spline.
  • Figure 5: Effect of detected nuclei count on the $rTRE$ of registration pipeline. It is evident that after exceeding 100 nuclei detections, $rTRE$ plateaus around $5\times10^{-3}$.