Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAM-OCTA: Prompting Segment-Anything for OCTA Image Segmentation

Xinrun Chen, Chengliang Wang, Haojian Ning, Shiying Li, Mei Shen

TL;DR

SAM-OCTA leverages a fine-tuned Segment Anything Model (SAM) via Low-Rank Adaptation (LoRA) to perform local OCTA image segmentation. By introducing prompt-point strategies (random and special annotation) and operating in global and local modes, the method achieves state-of-the-art performance on OCTA-500 for retinal vessels (RVs) and foveal avascular zone (FAZ), while also enabling artery-vein (AV) discrimination and capillary segmentation. The study systematically analyzes the role of prompt points, model scale, and fine-tuning, showing that prompt points chiefly enhance local segmentation and that larger ViT backbones generally yield better results, with fine-tuning being essential across settings. The approach offers a practical, parameter-efficient way to adapt a strong foundation model to OCTA-specific segmentation tasks, and the authors provide code for community use.

Abstract

Segmenting specific targets or biomarkers is necessary to analyze optical coherence tomography angiography (OCTA) images. Previous methods typically segment all the targets in an OCTA sample, such as retinal vessels (RVs). Although these methods perform well in accuracy and precision, OCTA analyses often focusing local information within the images which has not been fulfilled. In this paper, we propose a method called SAM-OCTA for local segmentation in OCTA images. The method fine-tunes a pre-trained segment anything model (SAM) using low-rank adaptation (LoRA) and utilizes prompt points for local RVs, arteries, and veins segmentation in OCTA. To explore the effect and mechanism of prompt points, we set up global and local segmentation modes with two prompt point generation strategies, namely random selection and special annotation. Considering practical usage, we conducted extended experiments with different model scales and analyzed the model performance before and after fine-tuning besides the general segmentation task. From comprehensive experimental results with the OCTA-500 dataset, our SAM-OCTA method has achieved state-of-the-art performance in common OCTA segmentation tasks related to RV and FAZ, and it also performs accurate segmentation of artery-vein and local vessels. The code is available at https://github.com/ShellRedia/SAM-OCTA-extend.

SAM-OCTA: Prompting Segment-Anything for OCTA Image Segmentation

TL;DR

SAM-OCTA leverages a fine-tuned Segment Anything Model (SAM) via Low-Rank Adaptation (LoRA) to perform local OCTA image segmentation. By introducing prompt-point strategies (random and special annotation) and operating in global and local modes, the method achieves state-of-the-art performance on OCTA-500 for retinal vessels (RVs) and foveal avascular zone (FAZ), while also enabling artery-vein (AV) discrimination and capillary segmentation. The study systematically analyzes the role of prompt points, model scale, and fine-tuning, showing that prompt points chiefly enhance local segmentation and that larger ViT backbones generally yield better results, with fine-tuning being essential across settings. The approach offers a practical, parameter-efficient way to adapt a strong foundation model to OCTA-specific segmentation tasks, and the authors provide code for community use.

Abstract

Segmenting specific targets or biomarkers is necessary to analyze optical coherence tomography angiography (OCTA) images. Previous methods typically segment all the targets in an OCTA sample, such as retinal vessels (RVs). Although these methods perform well in accuracy and precision, OCTA analyses often focusing local information within the images which has not been fulfilled. In this paper, we propose a method called SAM-OCTA for local segmentation in OCTA images. The method fine-tunes a pre-trained segment anything model (SAM) using low-rank adaptation (LoRA) and utilizes prompt points for local RVs, arteries, and veins segmentation in OCTA. To explore the effect and mechanism of prompt points, we set up global and local segmentation modes with two prompt point generation strategies, namely random selection and special annotation. Considering practical usage, we conducted extended experiments with different model scales and analyzed the model performance before and after fine-tuning besides the general segmentation task. From comprehensive experimental results with the OCTA-500 dataset, our SAM-OCTA method has achieved state-of-the-art performance in common OCTA segmentation tasks related to RV and FAZ, and it also performs accurate segmentation of artery-vein and local vessels. The code is available at https://github.com/ShellRedia/SAM-OCTA-extend.
Paper Structure (22 sections, 10 equations, 11 figures, 6 tables, 1 algorithm)

This paper contains 22 sections, 10 equations, 11 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. OCTA Segmentation Models
  4. Improvement of Segment Anything Model
  5. Artery-vein Segmentation
  6. Method
  7. Fine-tuning Image Encoder
  8. Prompt Points Generation Strategy
  9. Random Selection
  10. Special Annotation
  11. Experiments
  12. Datasets and Preprocessing
  13. Experimental Settings
  14. Results
  15. Random Selection
  16. ...and 7 more sections

Figures (11)

  • Figure 1: Schematic diagram illustrating the fine-tuning of SAM using OCTA samples.
  • Figure 2: OCTA structural diagram. (a) Three-dimensional volume rendering. (b) En-face projection. (c) B-scan with annotated anatomic layers: internal limiting membrane (ILM), outer plexiform layer (OPL), and Bruch’s membrane (BM). (d) An en-face OCTA sample of the OCTA-500 dataset.
  • Figure 3: Two prompt points generation modes. (a) Global mode. (b) Local mode.
  • Figure 4: Schematic of random selection prompt method. (a) Positive and negative regions generation. (b) Prompt points selection of increasing quantity. (c) Handling of irregular cases. (d) Special strategy for artery-vein segmentation in global mode.
  • Figure 5: Sparse annotation method for special points based on heatmaps.
  • ...and 6 more figures