SAM-OCTA2: Layer Sequence OCTA Segmentation with Fine-tuned Segment Anything Model 2

TL;DR

This work addresses the challenge of segmenting OCTA targets across 3D layer sequences by adapting a foundational segmentation model to OCTA data. It introduces SAM-OCTA2, which fine-tunes SAM 2 using LoRA on the image encoder, augmented with a memory-based fusion mechanism to track targets across layers, and combines a prompt-point strategy with a sparse layer annotation workflow. The method achieves competitive en-face segmentation performance and demonstrates effective layer-sequence tracking, supported by a novel 2D-to-3D annotation pipeline and ablation studies showing the benefits of frame- and point-based prompts. The approach offers a flexible, low-label-burden pathway for 3D OCTA analysis with practical implications for retinal disease assessment and 3D vascular reconstruction, with code available for community use.

Abstract

Segmentation of indicated targets aids in the precise analysis of optical coherence tomography angiography (OCTA) samples. Existing segmentation methods typically perform on 2D projection targets, making it challenging to capture the variance of segmented objects through the 3D volume. To address this limitation, the low-rank adaptation technique is adopted to fine-tune the Segment Anything Model (SAM) version 2, enabling the tracking and segmentation of specified objects across the OCTA scanning layer sequence. To further this work, a prompt point generation strategy in frame sequence and a sparse annotation method to acquire retinal vessel (RV) layer masks are proposed. This method is named SAM-OCTA2 and has been experimented on the OCTA-500 dataset. It achieves state-of-the-art performance in segmenting the foveal avascular zone (FAZ) on regular 2D en-face and effectively tracks local vessels across scanning layer sequences. The code is available at: https://github.com/ShellRedia/SAM-OCTA2.

Figures (5)

• Figure 1: The schematic diagram of the SAM-OCTA2 structure. The original model weights are frozen to preserve the semantic understanding and image processing capabilities through pre-training. The memory bank is essentially a queue and does not contain trainable parameters.
• Figure 2: The illustration of prompt point generation in the scanning layers of OCTA samples. Each vessel is represented by a distinct color. The purple vessel is selected as the segmentation target. The red regions surrounding vessels are designated to propose negative points.
• Figure 3: Illustration of sparse annotation for retinal vessel in scanning layers. We segment the regions where vessels appear instead of segmenting the RVs directly. This strategy utilizes existing annotations to enhance accuracy. The layer distributor randomly selects batches of potential RV scanning layers for manual annotation. For regions, blue represents the manually annotated, yellow indicates the model-predicted, and red denotes the modified. The predicted region is smoothed by the Gaussian filter.
• Figure 4: Segmentation samples of RV and FAZ on en-face OCTA images. The FAZ region has been enlarged for clearer observation, and the same applies to Fig.\ref{['Fig_ResultSamples_Sequence']} below.
• Figure 5: Samples of layer sequence segmentation with four frames. For simplicity, only three vessels are shown in the RV segmentation, distinguished by different colors. Note that each vessel is predicted separately, and the figure merges the results for visualization.