UCS: A Universal Model for Curvilinear Structure Segmentation

TL;DR

UCS tackles cross-domain curvilinear structure segmentation by adapting SAM with Sparse Adapters and an FFT-based Prompt Generation to inject curve-focused cues into each encoder block. A dual-branch mask decoder, comprising Hierarchical Feature Compression and Guidance Feature Compression, fuses multi-level encoder information and automatic guidance, paired with a composite loss to optimize mask quality. Extensive experiments on in-house and public datasets demonstrate state-of-the-art cross-domain generalization and open-set performance, with high parameter efficiency (≈2.65% trainable) and robustness to realistic degradations. The work establishes a strong benchmark for universal CSS and highlights avenues for further efficiency and multi-modal integration.

Abstract

Curvilinear structure segmentation (CSS) is essential in various domains, including medical imaging, landscape analysis, industrial surface inspection, and plant analysis. While existing methods achieve high performance within specific domains, their generalizability is limited. On the other hand, large-scale models such as Segment Anything Model (SAM) exhibit strong generalization but are not optimized for curvilinear structures. Existing adaptations of SAM primarily focus on general object segmentation and lack specialized design for CSS tasks. To bridge this gap, we propose the Universal Curvilinear structure Segmentation (UCS) model, which adapts SAM to CSS tasks while further enhancing its cross-domain generalization. UCS features a novel encoder architecture integrating a pretrained SAM encoder with two innovations: a Sparse Adapter, strategically inserted to inherit the pre-trained SAM encoder's generalization capability while minimizing the number of fine-tuning parameters, and a Prompt Generation module, which leverages Fast Fourier Transform with a high-pass filter to generate curve-specific prompts. Furthermore, the UCS incorporates a mask decoder that eliminates reliance on manual interaction through a dual-compression module: a Hierarchical Feature Compression module, which aggregates the outputs of the sampled encoder to enhance detail preservation, and a Guidance Feature Compression module, which extracts and compresses image-driven guidance features. Evaluated on a comprehensive multi-domain dataset, including an in-house dataset covering eight natural curvilinear structures, UCS demonstrates state-of-the-art generalization and open-set segmentation performance across medical, engineering, natural, and plant imagery, establishing a new benchmark for universal CSS. The source code is available at https://github.com/kylechuuuuu/UCS.

Figures (13)

• Figure 1: Comparison of F1-score across diverse segmentation models and datasets, where $^{\ast}$ denotes in-house dataset.
• Figure 2: Visualization results of curvilinear segmentation by UCS across various scenarios. These images are selected from our in-house dataset and represent key examples from eight scene categories. The (a) shows the input images, while the (b) presents the corresponding segmentation results.
• Figure 3: The proposed UCS model architecture. We freeze the image encoder and insert a learnable Sparse Adapter (SA) in specific encoder blocks, namely the 5th, 11th, 17th, and 23rd blocks, as shown in (a). The Prompt Generation (PG) module, illustrated in (b), integrates a Fast Fourier Transform with a high-pass filter to transform high frequency into dynamic prompts using a residual connection. The Hierarchical Feature Compression (HFC) module, shown in (c), compresses the outputs of the sampled encoders and adds them to the decoder layer. The Guidance Feature Compression (GFC) extracts image-driven guidance features. During training, the pretrained weights of the SAM encoder are frozen, and the rest of the model is trained.
• Figure 4: (a) Previous adapter approach (b) Our Sparse Adapter.
• Figure 5: Structure of the Adapter.