FishNet: Deep Neural Networks for Low-Cost Fish Stock Estimation

TL;DR

FishNet delivers an end-to-end, low-cost image-based solution for automated fish stock estimation by combining Mask R-CNN–driven detection/segmentation with species classification and per-fish length regression. Trained on a massive, citizen-science–generated dataset (300{,}000 images, 1.2 million fish, 163 species) and using fiduciary markers for scale, it achieves strong segmentation ($\text{IoU} \approx 92\%$ for fish), high species recognition ($\text{top-1} \approx 89\%$), and precise length estimation ($\text{MAE} = 2.3$ cm, $R^{2}=0.79$) on held-out data. The approach addresses a critical bottleneck in fisheries stock assessment by reducing reliance on expensive expert counting and enabling scalable, on-camera size and species estimation with a low-cost setup. This work demonstrates the practical impact of combining citizen-science data collection with robust CV pipelines for real-world, multi-species stock monitoring. The methods show promise for broader adoption in developing regions, where cost-effective, accurate stock assessment is essential for sustainable fisheries management.

Abstract

Fish stock assessment often involves manual fish counting by taxonomy specialists, which is both time-consuming and costly. We propose FishNet, an automated computer vision system for both taxonomic classification and fish size estimation from images captured with a low-cost digital camera. The system first performs object detection and segmentation using a Mask R-CNN to identify individual fish from images containing multiple fish, possibly consisting of different species. Then each fish species is classified and the length is predicted using separate machine learning models. To develop the model, we use a dataset of 300,000 hand-labeled images containing 1.2M fish of 163 different species and ranging in length from 10cm to 250cm, with additional annotations and quality control methods used to curate high-quality training data. On held-out test data sets, our system achieves a 92% intersection over union on the fish segmentation task, a 89% top-1 classification accuracy on single fish species classification, and a 2.3cm mean absolute error on the fish length estimation task.

Figures (11)

• Figure 1: Fishers in Indonesia on their boats photographing their catch on a standard color-coded measuring board. Source: Ed Wray, for Yayasan Konservasi Alam Nusantara.
• Figure 2: FishNet's training process, starting from data labeling to achieving high-accuracy fish classification and size estimation. It shows the data flow between datasets and models.
• Figure 3: Example of an image segmented using Detectron2 to identify both the fish and the 10 cm colored rectangles used as fiduciary markers. The original image is in color, but has been shown in grayscale to show the segmentation. Source: Yayasan Konservasi Alam Nusantara.
• Figure 4: Image of a fish along with the yellow and blue reference color boxes on either side of the boards. In this example, a human annotator has outlined these objects using the VGG Annotation tool. Source: Yayasan Konservasi Alam Nusantara.
• Figure 5: Heat map of detected vs. ground truth fish count in multi-fish images. Diagonal entries indicate where the model correctly detects and segments the number of fish