The Coralscapes Dataset: Semantic Scene Understanding in Coral Reefs

TL;DR

The paper tackles the need for scalable, high-resolution reef monitoring by introducing Coralscapes, the first general-purpose dense semantic segmentation dataset for coral reefs. It provides 2075 expert-annotated images across 39 classes (174k masks) with a Cityscapes-like structure to enable fair benchmarking of segmentation models in underwater scenes. Through extensive benchmarking, the authors show that transfer learning from Coralscapes improves performance on smaller, domain-specific reef datasets and demonstrate practical downstream applications, including cross-domain transfer, COTS detection, and underwater 3D mapping. The dataset is poised to catalyze scalable reef surveying and robotics, with planned expansions to broaden diversity and add finer-grained annotations.

Abstract

Coral reefs are declining worldwide due to climate change and local stressors. To inform effective conservation or restoration, monitoring at the highest possible spatial and temporal resolution is necessary. Conventional coral reef surveying methods are limited in scalability due to their reliance on expert labor time, motivating the use of computer vision tools to automate the identification and abundance estimation of live corals from images. However, the design and evaluation of such tools has been impeded by the lack of large high quality datasets. We release the Coralscapes dataset, the first general-purpose dense semantic segmentation dataset for coral reefs, covering 2075 images, 39 benthic classes, and 174k segmentation masks annotated by experts. Coralscapes has a similar scope and the same structure as the widely used Cityscapes dataset for urban scene segmentation, allowing benchmarking of semantic segmentation models in a new challenging domain which requires expert knowledge to annotate. We benchmark a wide range of semantic segmentation models, and find that transfer learning from Coralscapes to existing smaller datasets consistently leads to state-of-the-art performance. Coralscapes will catalyze research on efficient, scalable, and standardized coral reef surveying methods based on computer vision, and holds the potential to streamline the development of underwater ecological robotics.

Figures (14)

• Figure 1: Number of annotated segmentation masks per class in the Coralscapes dataset splits for each of the 39 classes (shown with linear proportions on logarithmic scale).
• Figure 2: Histogram of the image area covered by annotations (top) and of the number of polygons present per image (bottom), highlighting the complexity of semantic segmentation in Coralscapes, compared to the CoralSCOP dataset zheng2024coralscop. Further comparison to the CoralSCOP dataset can be found in Appendix \ref{['appendix:coralscop']}.
• Figure 3: Median size (in pixels) of an annotated polygon for the classes of Coralscapes, plotted against the number of annotated polygons. This highlights the challenge of segmenting classes that require the global image structure to segment correctly, as well as small fine-grained classes in the same dataset.
• Figure 4: Qualitative samples from the Coralscapes test set. Additional samples provided in Appendix \ref{['appendix:additional_samples']}.
• Figure 5: Using masks obtained from Coralscapes to mask out unwanted classes can alleviate artifacts from dense 3D reconstruction and from novel view synthesis such as 3DGS.