Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Informative Sampling using Semantic Features in Underwater Environments

Shrutika Vishal Thengane, Yu Xiang Tan, Marcel Bartholomeus Prasetyo, Malika Meghjani

TL;DR

This work tackles data volume and bandwidth limitations in underwater exploration by presenting Semantic Online Informative Sampling (SON-IS), which integrates semantic features from a fine-tuned DETR detector with an online sampling pipeline inspired by ROST. The method emphasizes semantic uniqueness rather than visual novelty, enabling AUVs to prioritize frames containing distinct marine life. A novel SRUM metric assesses alignment between automated samples and human selections, and a human study validates semantic usefulness. Empirical results show SON-IS outperforms the baseline on semantic representativeness, suggesting a practical pathway for efficient, semantically aware underwater monitoring.

Abstract

The underwater world remains largely unexplored, with Autonomous Underwater Vehicles (AUVs) playing a crucial role in sub-sea explorations. However, continuous monitoring of underwater environments using AUVs can generate a significant amount of data. In addition, sending live data feed from an underwater environment requires dedicated on-board data storage options for AUVs which can hinder requirements of other higher priority tasks. Informative sampling techniques offer a solution by condensing observations. In this paper, we present a semantically-aware online informative sampling (ON-IS) approach which samples an AUV's visual experience in real-time. Specifically, we obtain visual features from a fine-tuned object detection model to align the sampling outcomes with the desired semantic information. Our contributions are (a) a novel Semantic Online Informative Sampling (SON-IS) algorithm, (b) a user study to validate the proposed approach and (c) a novel evaluation metric to score our proposed algorithm with respect to the suggested samples by human subjects

Online Informative Sampling using Semantic Features in Underwater Environments

TL;DR

This work tackles data volume and bandwidth limitations in underwater exploration by presenting Semantic Online Informative Sampling (SON-IS), which integrates semantic features from a fine-tuned DETR detector with an online sampling pipeline inspired by ROST. The method emphasizes semantic uniqueness rather than visual novelty, enabling AUVs to prioritize frames containing distinct marine life. A novel SRUM metric assesses alignment between automated samples and human selections, and a human study validates semantic usefulness. Empirical results show SON-IS outperforms the baseline on semantic representativeness, suggesting a practical pathway for efficient, semantically aware underwater monitoring.

Abstract

The underwater world remains largely unexplored, with Autonomous Underwater Vehicles (AUVs) playing a crucial role in sub-sea explorations. However, continuous monitoring of underwater environments using AUVs can generate a significant amount of data. In addition, sending live data feed from an underwater environment requires dedicated on-board data storage options for AUVs which can hinder requirements of other higher priority tasks. Informative sampling techniques offer a solution by condensing observations. In this paper, we present a semantically-aware online informative sampling (ON-IS) approach which samples an AUV's visual experience in real-time. Specifically, we obtain visual features from a fine-tuned object detection model to align the sampling outcomes with the desired semantic information. Our contributions are (a) a novel Semantic Online Informative Sampling (SON-IS) algorithm, (b) a user study to validate the proposed approach and (c) a novel evaluation metric to score our proposed algorithm with respect to the suggested samples by human subjects
Paper Structure (18 sections, 1 equation, 3 figures, 3 tables, 1 algorithm)

This paper contains 18 sections, 1 equation, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Informative Sampling
  4. Online Sampling
  5. Offline Sampling
  6. Semantic Recognition
  7. Proposed Methodology
  8. Feature Extraction Module
  9. Online Informative Sampling Module
  10. Evaluation Metric
  11. Experimental Setup
  12. Finetuning Dataset
  13. Evaluation dataset
  14. Results and Discussion
  15. Collection of Human-picked Samples
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Overview of SON-IS algorithm comprising of the feature extraction and online sampling modules.
  • Figure 2: The left image is from the Brackish dataset pedersen2019brackish used for fine-tuning and right image is from the evaluation dataset solodive.
  • Figure 3: Qualitative study of ROST sample ($A$), SON-IS sample ($H$, Our Method) and human-picked samples ($B$, $C$, $D$, $E$, $F$ and $G$) .