Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fidelity-Imposed Displacement Editing for the Learn2Reg 2024 SHG-BF Challenge

Jiacheng Wang, Xiang Chen, Renjiu Hu, Rongguang Wang, Jiazheng Wang, Min Liu, Yaonan Wang, Hang Zhang

TL;DR

A novel multi-modal registration framework that employs fidelity-imposed displacement editing to address large discrepancies between SHG and BF images is proposed and secured the 1st place on the online leaderboard.

Abstract

Co-examination of second-harmonic generation (SHG) and bright-field (BF) microscopy enables the differentiation of tissue components and collagen fibers, aiding the analysis of human breast and pancreatic cancer tissues. However, large discrepancies between SHG and BF images pose challenges for current learning-based registration models in aligning SHG to BF. In this paper, we propose a novel multi-modal registration framework that employs fidelity-imposed displacement editing to address these challenges. The framework integrates batch-wise contrastive learning, feature-based pre-alignment, and instance-level optimization. Experimental results from the Learn2Reg COMULISglobe SHG-BF Challenge validate the effectiveness of our method, securing the 1st place on the online leaderboard.

Fidelity-Imposed Displacement Editing for the Learn2Reg 2024 SHG-BF Challenge

TL;DR

A novel multi-modal registration framework that employs fidelity-imposed displacement editing to address large discrepancies between SHG and BF images is proposed and secured the 1st place on the online leaderboard.

Abstract

Co-examination of second-harmonic generation (SHG) and bright-field (BF) microscopy enables the differentiation of tissue components and collagen fibers, aiding the analysis of human breast and pancreatic cancer tissues. However, large discrepancies between SHG and BF images pose challenges for current learning-based registration models in aligning SHG to BF. In this paper, we propose a novel multi-modal registration framework that employs fidelity-imposed displacement editing to address these challenges. The framework integrates batch-wise contrastive learning, feature-based pre-alignment, and instance-level optimization. Experimental results from the Learn2Reg COMULISglobe SHG-BF Challenge validate the effectiveness of our method, securing the 1st place on the online leaderboard.

Paper Structure

This paper contains 12 sections, 7 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Features prealignment with contrastive learning
  5. Test-time Instance Optimization
  6. Multimodal Similarities: Mutual Information
  7. Experiments and Results
  8. Dataset
  9. Data Preprocessing and Evaluation Metrics
  10. Quantitative Results
  11. Qualitative Results
  12. Conclusion

Figures (5)

  • Figure 1: Illustration of our proposed batch-wise noise contrastive estimation (B-NCE) loss, which aggregates pixel-level information across patches, compared to traditional InfoNCE loss that operates directly on patch-level information.
  • Figure 2: Overview of our proposed framework, which follows three main steps: feature extraction, XFeat feature matching, and ConvexAdam fine-tuning.
  • Figure 3: XFeat feature detection/description/matching in SHG/BF images and the Tiramisu feature images.
  • Figure 4: Qualitative results on the validation set. From left to right: warped BF image overlaid on SHG image, SHG image (fixed) shown in Virdis colormap, BF image (moving), and warped BF image.
  • Figure 5: Impact of the number of optimization iterations on the mean TRE. Our method achieves consistent performance, with the best result at 30 iterations.