Novel OCT mosaicking pipeline with Feature- and Pixel-based registration

TL;DR

This paper tackles OCT mosaic stitching under noisy, deformed conditions by introducing a coarse-to-fine pipeline that blends learning-based feature matching (LightGlue with SuperPoint) and pixel-based deformable registration (SyN), connected by a novel feature-image affine bridge. It further uses the Segment Anything Model (SAM) with feature-derived prompts to enable unsupervised mosaic validation. Evaluations on in-house 9-field OCT data and the OCTA-500 synthetic dataset demonstrate superior accuracy and efficiency over baselines, and the authors release their evaluation tool publicly. The approach offers a practical, unsupervised framework for robust wide-field OCT panoramas and provides a valuable validation paradigm for medical image mosaicking.

Abstract

High-resolution Optical Coherence Tomography (OCT) images are crucial for ophthalmology studies but are limited by their relatively narrow field of view (FoV). Image mosaicking is a technique for aligning multiple overlapping images to obtain a larger FoV. Current mosaicking pipelines often struggle with substantial noise and considerable displacement between the input sub-fields. In this paper, we propose a versatile pipeline for stitching multi-view OCT/OCTA \textit{en face} projection images. Our method combines the strengths of learning-based feature matching and robust pixel-based registration to align multiple images effectively. Furthermore, we advance the application of a trained foundational model, Segment Anything Model (SAM), to validate mosaicking results in an unsupervised manner. The efficacy of our pipeline is validated using an in-house dataset and a large public dataset, where our method shows superior performance in terms of both accuracy and computational efficiency. We also made our evaluation tool for image mosaicking and the corresponding pipeline publicly available at \url{https://github.com/MedICL-VU/OCT-mosaicking}.

Figures (4)

• Figure 1: Four-stage OCT mosaicking pipeline. Stage I displays feature matching: the moving image in blue and the reference in purple, and 5 sample matched features. Stage II shows the features-intensity affine transform applied to the moving image. In Stage III, we use deformable registration to refine the alignment. In Stage IV, the final mosaic image is evaluated by SAM.
• Figure 2: Left: Original images with the central (purple) field serving as the fixed image. Right: Mosaicking result with a larger FoV. The contours of each sub-field are color-coded.
• Figure 3: (A) OCTA-500 Synthetic dataset: known random affine transform and elastic deformation were applied to the cropped fields (2 fields shown in dashed lines). (B) Rotation estimation error (degrees) and computation time (seconds) of each feature matching method. Each dot represents an individual subject's result, color-coded by methods. The highlighted quadrant shows LightGlue is the most desirable choice, combining high accuracy and efficiency. (C) Qualitative feature point comparison. LightGlue focuses on sparse but representative points, whereas LoFTR yields a dense grid of features that may not be as informative.
• Figure 4: (A) Quantitative comparison in the 9-field dataset: Light colors (MAGSAC and Affine) represent coarse stage results, while dark colors (MAGSAC+SyN and Affine+SyN) represent the refinement stage. Our method (Affine+SyN) outperformed MAGSAC in all three metrics at both stages. (B) Qualitative comparison, with pink arrows highlighting noteworthy differences. The intensity difference between the mosaic and reference images ($\Delta = |I_{ref}-I_{mosaic}|$) in the dashed region is represented using a blue-to-red color map. (C) SAM segmentation from the mosaic is compared against the reference image segmentation. Blue areas indicate segmentation agreement, while orange regions reveal discrepancies. Dice score for each method is reported.