A Multi-purpose Tracking Framework for Salmon Welfare Monitoring in Challenging Environments

TL;DR

The paper tackles automated salmon welfare monitoring in challenging underwater net pens, where occlusion and appearance similarity impede conventional trackers. It introduces BoostCompTrack, a pose-based, multi-scale tracking framework with TurnModule and CrowdedModule to preserve identities across crowded and turning scenes, and it demonstrates tailbeat wavelength estimation from body-part trajectories. The authors create three datasets (CS, TS, TBW) to evaluate crowded tracking, turning, and tailbeat analysis, and show that BoostCompTrack outperforms BoostTrack in critical scenarios while enabling direct welfare indicators such as tailbeat wavelength. This approach offers a scalable, low-overhead path toward multi-indicator welfare monitoring in industrial aquaculture, with potential for long-term re-identification using body-part textures.

Abstract

Computer Vision (CV)-based continuous, automated and precise salmon welfare monitoring is a key step toward reduced salmon mortality and improved salmon welfare in industrial aquaculture net pens. Available CV methods for determining welfare indicators focus on single indicators and rely on object detectors and trackers from other application areas to aid their welfare indicator calculation algorithm. This comes with a high resource demand for real-world applications, since each indicator must be calculated separately. In addition, the methods are vulnerable to difficulties in underwater salmon scenes, such as object occlusion, similar object appearance, and similar object motion. To address these challenges, we propose a flexible tracking framework that uses a pose estimation network to extract bounding boxes around salmon and their corresponding body parts, and exploits information about the body parts, through specialized modules, to tackle challenges specific to underwater salmon scenes. Subsequently, the high-detail body part tracks are employed to calculate welfare indicators. We construct two novel datasets assessing two salmon tracking challenges: salmon ID transfers in crowded scenes and salmon ID switches during turning. Our method outperforms the current state-of-the-art pedestrian tracker, BoostTrack, for both salmon tracking challenges. Additionally, we create a dataset for calculating salmon tail beat wavelength, demonstrating that our body part tracking method is well-suited for automated welfare monitoring based on tail beat analysis. Datasets and code are available at https://github.com/espenbh/BoostCompTrack.

Figures (6)

• Figure 1: A visualization of our proposed pipeline BoostCompTrack. Bounding boxes around salmon and salmon body parts are detected by a top-down pose model, before each salmon component is tracked separately and fused in a way that prevents certain challenges in underwater salmon tracking. The resulting tracks of salmon and salmon body parts can be leveraged for a diverse set of tasks, including segmentation mask generation with Segment Anything Kirillov2023 (as demonstrated in the middle image on the right) and tail beat wavelength estimation (as visualized in the top image on the right).
• Figure 2: Annotated images from the TailbeatWavelength training dataset (top), CrowdedSalmon validation dataset (middle) and TurnSalmon validation dataset (bottom). All images from the CS dataset were rotated 135 degrees prior to processing to correct for the orientation of the raw video, which was captured at a 135-degree angle relative to the water surface.
• Figure 3: Two consecutive extreme salmon poses, annotated with bounding boxes around all salmon components, white lines between selected body parts , and a red circle around the intersection point (IP). Additionally, the top frame is annotated with the points used to calculate IP. The contrast and brightness of the images are enhanced for visibility.
• Figure 4: Number of salmon events and body part events for different IoU thresholds. Note that the y-axis range of the plots changes since each plot displays a different event and object group (salmon and body parts). BT = BoostTrack, BCT = BoostCompTrack.
• Figure 5: Number of matches for turning salmon in the TurnSalmon validation dataset. hl indicates the hidden length setting. BT = BoostTrack, BCT = BoostCompTrack. all uses all BCT modules; turn uses only the TurnModule. The horizontal axis specifies the IoU setting of the tracker.