Geometric Feature Prompting of Image Segmentation Models

TL;DR

The paper tackles automated segmentation of fine plant root structures in rhizotron/minirhizotron imagery, a task hindered by labeling costs and image heterogeneity. It introduces Geometric Prompting (GeomPrompt), a method that harvests multiscale ridge detection to generate a sparse set of prompt points colocated with root-like features and feeds them into SAM (Segment Anything Model). By defining ridge strength AL = t^{2\gamma}((Lxx-Lyy)^2 + 4 Lxy^2) on a scale-space representation L(x,y;t) and aggregating ridge salience A(Gamma) along ridge curves, the authors allocate prompts proportionally to salient features, improving segmentation efficiency. On a subset of PRMI switchgrass minirhizotron data, GeomPrompt+SAM achieves higher true positive rates with lower false positives at low prompt densities compared to uniform grids, while maintaining competitive performance at higher prompt densities; an open-source geomprompt package facilitates integration with SAM. Overall, the work demonstrates that non-learned geometric cues can guide foundation-model prompts to task-relevant structures, enabling rapid adaptation for specialized segmentation tasks and suggesting broad applicability to SAM 2 and other prompt-based segmenters.

Abstract

Advances in machine learning, especially the introduction of transformer architectures and vision transformers, have led to the development of highly capable computer vision foundation models. The segment anything model (known colloquially as SAM and more recently SAM 2), is a highly capable foundation model for segmentation of natural images and has been further applied to medical and scientific image segmentation tasks. SAM relies on prompts -- points or regions of interest in an image -- to generate associated segmentations. In this manuscript we propose the use of a geometrically motivated prompt generator to produce prompt points that are colocated with particular features of interest. Focused prompting enables the automatic generation of sensitive and specific segmentations in a scientific image analysis task using SAM with relatively few point prompts. The image analysis task examined is the segmentation of plant roots in rhizotron or minirhizotron images, which has historically been a difficult task to automate. Hand annotation of rhizotron images is laborious and often subjective; SAM, initialized with GeomPrompt local ridge prompts has the potential to dramatically improve rhizotron image processing. The authors have concurrently released an open source software suite called geomprompt https://pypi.org/project/geomprompt/ that can produce point prompts in a format that enables direct integration with the segment-anything package.

Figures (3)

• Figure 1: A $1024\times 1024$ minirhizotron image sample with \ref{['ex1:a']} 1001 ridge point prompts and \ref{['ex1:b']} 1024 uniform grid point prompts. By prompting SAM in with local ridge like features, resulting good quality (predicted IoU $>0.75$) segmentation masks are much more focused on objects exhibiting those features \ref{['ex1:c']} than masks generated with naive grid points \ref{['ex1:d']}.
• Figure 2: An example of benchmark minirhizotron image (left), a contrast enhanced version (center), and an the accompanying annotation of root segments. Significant ridge-like features of varying scales are not included in the reference annotation, which we note not to impugn annotation quality in this particular case but rather to illustrate that the geomprompt + SAM objective (segment ridge-like features) differs from the benchmark analysis objective (segment plant roots of interest). Also, contrast enhancement is used here to illustrate faint fine features of potential interest, but was not used in our experimental analysis.
• Figure 3: Examples of 64 point density grid prompted SAM segmentation (left), geomprompt SAM segmentation, and the benchmark reference annotation, overlaid on a contrast enhanced image detail.