Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OysterNet: Enhanced Oyster Detection Using Simulation

Xiaomin Lin, Nitin J. Sanket, Nare Karapetyan, Yiannis Aloimonos

TL;DR

A novel method to mathematically model oysters and render images of oysters in simulation to boost the detection performance with minimal real data is presented and shows that using underlying geometrical properties of objects can help to enhance recognition task accuracy on limited datasets successfully.

Abstract

Oysters play a pivotal role in the bay living ecosystem and are considered the living filters for the ocean. In recent years, oyster reefs have undergone major devastation caused by commercial over-harvesting, requiring preservation to maintain ecological balance. The foundation of this preservation is to estimate the oyster density which requires accurate oyster detection. However, systems for accurate oyster detection require large datasets obtaining which is an expensive and labor-intensive task in underwater environments. To this end, we present a novel method to mathematically model oysters and render images of oysters in simulation to boost the detection performance with minimal real data. Utilizing our synthetic data along with real data for oyster detection, we obtain up to 35.1% boost in performance as compared to using only real data with our OysterNet network. We also improve the state-of-the-art by 12.7%. This shows that using underlying geometrical properties of objects can help to enhance recognition task accuracy on limited datasets successfully and we hope more researchers adopt such a strategy for hard-to-obtain datasets.

OysterNet: Enhanced Oyster Detection Using Simulation

TL;DR

A novel method to mathematically model oysters and render images of oysters in simulation to boost the detection performance with minimal real data is presented and shows that using underlying geometrical properties of objects can help to enhance recognition task accuracy on limited datasets successfully.

Abstract

Oysters play a pivotal role in the bay living ecosystem and are considered the living filters for the ocean. In recent years, oyster reefs have undergone major devastation caused by commercial over-harvesting, requiring preservation to maintain ecological balance. The foundation of this preservation is to estimate the oyster density which requires accurate oyster detection. However, systems for accurate oyster detection require large datasets obtaining which is an expensive and labor-intensive task in underwater environments. To this end, we present a novel method to mathematically model oysters and render images of oysters in simulation to boost the detection performance with minimal real data. Utilizing our synthetic data along with real data for oyster detection, we obtain up to 35.1% boost in performance as compared to using only real data with our OysterNet network. We also improve the state-of-the-art by 12.7%. This shows that using underlying geometrical properties of objects can help to enhance recognition task accuracy on limited datasets successfully and we hope more researchers adopt such a strategy for hard-to-obtain datasets.
Paper Structure (13 sections, 7 equations, 7 figures, 2 tables)

This paper contains 13 sections, 7 equations, 7 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Synthetic Oyster Generation
  4. Geometric Modelling Of Oysters
  5. Synthetic Image Generation
  6. Simulation Image Rendering
  7. Domain Adaptation
  8. Details of Training the CUT
  9. Experiments And Results
  10. Description of Datasets
  11. Experimental Results
  12. Ablation Studies
  13. CONCLUSIONS

Figures (7)

  • Figure 1: Each row left to right: Input image, output of the network when trained using only real data, output of the network (which we call OysterNet) when trained using real data augmented with our synthetic data. Yellow represents the oyster segmentation ground truth and the blue is the predicted segmentation result. Notice how the number of false positives and false negatives drop significantly when the training data is augmented with our synthetic data, All the images in this paper are best seen in color on a computer screen at 200% zoom.
  • Figure 2: An overview of our approach: The proposed geometric model is used to generate synthetic images which are further fed into a Generative Adversarial Network to enable sim-to-real transfer (domain adaptation) by generating photorealistic oyster images. We then combine the synthetic data with real data to train a OysterNet for oyster detection.
  • Figure 3: Steps in the geometric modelling of an oyster: (a) Sample image of a real oyster shell, (b) 3D scan of a real oyster, (c) Splines fit to the oyster's bottom layer to model it (each color represents a single spline), (d) Simplified model of one stratified layer of the oyster (single layer of spline curve $S(t)$), (e) all layers of $S^\alpha(t)$, (f) generated 3D model of oyster, (g) final generated 3D model of oyster with added real oyster texture.
  • Figure 4: (a) One of the synthetic models generated from Sec. \ref{['section:problem_formulation']}-A), (b) Synthetic model with real oyster texture added, (c) image of an oyster farm with 50 synthetic oysters generated in Blender$^{\text{TM}}$, (d) masks for oysters in (c).
  • Figure 5: Domain Adaptation to perform sim-to-real transfer. (a) A single sample of the simulated oyster, (b) A single sample of the real oyster, (c) Simulated oyster farm, (d) Synthetic oyster farm domain adapted to real world for photorealism.
  • ...and 2 more figures