Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TopoLoRA-SAM: Topology-Aware Parameter-Efficient Adaptation of Foundation Segmenters for Thin-Structure and Cross-Domain Binary Semantic Segmentation

Salim Khazem

TL;DR

TopoLoRA-SAM introduces a topology-aware, parameter-efficient framework for adapting a foundation segmentation model (SAM) to binary semantic segmentation across thin structures and cross-domain modalities. By freezing the SAM ViT-B encoder and injecting Low-Rank Adaptation (LoRA) into the FFN layers, complemented by a lightweight spatial adapter and topology-aware supervision via differentiable clDice, the method attains state-of-the-art retina-average Dice and overall Dice while updating only about 5.2% of parameters. Across retinal vessels, polyps, and SAR imagery, the approach demonstrates strong boundary and connectivity preservation (BFScore, clDice) and robustness on challenging CHASE_DB1, with substantial parameter efficiency compared to fully fine-tuned baselines. The work offers practical, reproducible code and highlights a clear contribution: training-efficient, topology-conscious adaptation of large vision models for domain-specific binary segmentation, with potential extensions to video, 3D, and multi-class tasks.

Abstract

Foundation segmentation models such as the Segment Anything Model (SAM) exhibit strong zero-shot generalization through large-scale pretraining, but adapting them to domain-specific semantic segmentation remains challenging, particularly for thin structures (e.g., retinal vessels) and noisy modalities (e.g., SAR imagery). Full fine-tuning is computationally expensive and risks catastrophic forgetting. We propose \textbf{TopoLoRA-SAM}, a topology-aware and parameter-efficient adaptation framework for binary semantic segmentation. TopoLoRA-SAM injects Low-Rank Adaptation (LoRA) into the frozen ViT encoder, augmented with a lightweight spatial convolutional adapter and optional topology-aware supervision via differentiable clDice. We evaluate our approach on five benchmarks spanning retinal vessel segmentation (DRIVE, STARE, CHASE\_DB1), polyp segmentation (Kvasir-SEG), and SAR sea/land segmentation (SL-SSDD), comparing against U-Net, DeepLabV3+, SegFormer, and Mask2Former. TopoLoRA-SAM achieves the best retina-average Dice and the best overall average Dice across datasets, while training only \textbf{5.2\%} of model parameters ($\sim$4.9M). On the challenging CHASE\_DB1 dataset, our method substantially improves segmentation accuracy and robustness, demonstrating that topology-aware parameter-efficient adaptation can match or exceed fully fine-tuned specialist models. Code is available at : https://github.com/salimkhazem/Seglab.git

TopoLoRA-SAM: Topology-Aware Parameter-Efficient Adaptation of Foundation Segmenters for Thin-Structure and Cross-Domain Binary Semantic Segmentation

TL;DR

TopoLoRA-SAM introduces a topology-aware, parameter-efficient framework for adapting a foundation segmentation model (SAM) to binary semantic segmentation across thin structures and cross-domain modalities. By freezing the SAM ViT-B encoder and injecting Low-Rank Adaptation (LoRA) into the FFN layers, complemented by a lightweight spatial adapter and topology-aware supervision via differentiable clDice, the method attains state-of-the-art retina-average Dice and overall Dice while updating only about 5.2% of parameters. Across retinal vessels, polyps, and SAR imagery, the approach demonstrates strong boundary and connectivity preservation (BFScore, clDice) and robustness on challenging CHASE_DB1, with substantial parameter efficiency compared to fully fine-tuned baselines. The work offers practical, reproducible code and highlights a clear contribution: training-efficient, topology-conscious adaptation of large vision models for domain-specific binary segmentation, with potential extensions to video, 3D, and multi-class tasks.

Abstract

Foundation segmentation models such as the Segment Anything Model (SAM) exhibit strong zero-shot generalization through large-scale pretraining, but adapting them to domain-specific semantic segmentation remains challenging, particularly for thin structures (e.g., retinal vessels) and noisy modalities (e.g., SAR imagery). Full fine-tuning is computationally expensive and risks catastrophic forgetting. We propose \textbf{TopoLoRA-SAM}, a topology-aware and parameter-efficient adaptation framework for binary semantic segmentation. TopoLoRA-SAM injects Low-Rank Adaptation (LoRA) into the frozen ViT encoder, augmented with a lightweight spatial convolutional adapter and optional topology-aware supervision via differentiable clDice. We evaluate our approach on five benchmarks spanning retinal vessel segmentation (DRIVE, STARE, CHASE\_DB1), polyp segmentation (Kvasir-SEG), and SAR sea/land segmentation (SL-SSDD), comparing against U-Net, DeepLabV3+, SegFormer, and Mask2Former. TopoLoRA-SAM achieves the best retina-average Dice and the best overall average Dice across datasets, while training only \textbf{5.2\%} of model parameters (4.9M). On the challenging CHASE\_DB1 dataset, our method substantially improves segmentation accuracy and robustness, demonstrating that topology-aware parameter-efficient adaptation can match or exceed fully fine-tuned specialist models. Code is available at : https://github.com/salimkhazem/Seglab.git
Paper Structure (38 sections, 6 equations, 4 figures, 2 tables)

This paper contains 38 sections, 6 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Semantic segmentation architectures.
  4. Foundation models for segmentation.
  5. SAM adaptation for specialized domains.
  6. Parameter-efficient fine-tuning.
  7. Topology-aware losses and metrics.
  8. Problem Formulation
  9. SAM Architecture and Prompt-Free Decoding
  10. Low-Rank Adaptation (LoRA)
  11. Spatial Convolutional Adapter
  12. Topology-Aware Training Objective
  13. Binary cross-entropy (BCE)
  14. Soft Dice loss
  15. Centerline Dice (clDice) shit2021cldice
  16. ...and 23 more sections

Figures (4)

  • Figure 1: TopoLoRA-SAM architecture overview. We freeze the SAM ViT-B image encoder and inject LoRA modules (red) into the feed-forward network (FFN) layers of each transformer block. A lightweight depthwise-separable convolutional adapter (green) refines the high-resolution embedding tensor before mask decoding. Training uses a topology-aware loss combining BCE, Dice, and clDice.
  • Figure 2: Thin-structure performance summary across retinal datasets (DRIVE, STARE, CHASE_DB1). TopoLoRA-SAM achieves top-tier performance on Dice, clDice, and BFScore while training 10$\times$ fewer parameters.
  • Figure 3: Qualitative comparison across all benchmark datasets. Each row shows a representative sample from one dataset: retinal vessel segmentation (DRIVE, STARE, CHASE_DB1), polyp segmentation (Kvasir-SEG), and SAR sea/land segmentation (SL-SSDD). Columns display input images, ground truth masks, and predictions from five methods. TopoLoRA-SAM (Ours) demonstrates superior preservation of thin vessel structures and fine-grained boundaries compared to baselines, particularly visible in the retinal datasets where thin peripheral vessels are better captured. On CHASE_DB1, our method maintains vessel connectivity while other methods show fragmentation. Best viewed zoomed in.
  • Figure 4: Parameter efficiency trade-off on retinal datasets. TopoLoRA-SAM achieves Pareto-optimal performance: competitive Dice with 10$\times$ fewer trainable parameters than Mask2Former.