Evaluation of Segment Anything Model 2: The Role of SAM2 in the Underwater Environment
Shijie Lian, Hua Li
TL;DR
The paper assesses SAM2 for underwater instance segmentation using the UIIS and USIS10K benchmarks, comparing it to SAM and EfficientSAM across GT Bbox, 1 Point, and 3 Point prompts. It demonstrates that SAM2 achieves strongest performance with ground-truth prompts, notably yielding up to a $4.8$ AP $mAP$ gain (e.g., SAM2-Hiera-Large) and a substantial speed advantage ($\approx5\times$) over prior models, while automatic prompting markedly reduces accuracy. A thorough end-to-end evaluation with $32^2$ prompts and Hungarian matching shows pronounced degradation in fully automatic mode, with FPS around $1.5$ and $mAP$ gaps of $-26.1$ to $-24.9$ AP compared to SAM-ViT-Huge. The work highlights the need for a robust prompt-generation module and suggests SAM2 could become a valuable annotation tool for underwater video segmentation, supported by the public evaluation code at https://github.com/LiamLian0727/UnderwaterSAM2Eval.
Abstract
With breakthroughs in large-scale modeling, the Segment Anything Model (SAM) and its extensions have been attempted for applications in various underwater visualization tasks in marine sciences, and have had a significant impact on the academic community. Recently, Meta has further developed the Segment Anything Model 2 (SAM2), which significantly improves running speed and segmentation accuracy compared to its predecessor. This report aims to explore the potential of SAM2 in marine science by evaluating it on the underwater instance segmentation benchmark datasets UIIS and USIS10K. The experiments show that the performance of SAM2 is extremely dependent on the type of user-provided prompts. When using the ground truth bounding box as prompt, SAM2 performed excellently in the underwater instance segmentation domain. However, when running in automatic mode, SAM2's ability with point prompts to sense and segment underwater instances is significantly degraded. It is hoped that this paper will inspire researchers to further explore the SAM model family in the underwater domain. The results and evaluation codes in this paper are available at https://github.com/LiamLian0727/UnderwaterSAM2Eval.