Benchmarking Fish Dataset and Evaluation Metric in Keypoint Detection -- Towards Precise Fish Morphological Assessment in Aquaculture Breeding

TL;DR

This work tackles the challenge of precise morphological phenotyping in aquaculture by releasing FishPhenoKey, a large, multi-species dataset with 22 keypoints enabling 23 phenotypes. It introduces the Percentage of Measured Phenotype (PMP) as a phenotype-aware evaluation metric and the Anatomically-Calibrated Regularization (ACR) loss to inject anatomical priors into keypoint detection, improving both localization and downstream phenotype measurement. Experiments show PMP offers robust per-keypoint assessment over OKS/PCK and that ACR enhances performance across backbones and species, yielding substantial gains in morphology accuracy (e.g., $R^2$ rising to 0.96 for a target phenotype). The dataset and code pave the way for more reliable, scalable phenotypic analysis in aquaculture breeding and genetics, supporting sustainable practices and diverse research avenues.

Abstract

Accurate phenotypic analysis in aquaculture breeding necessitates the quantification of subtle morphological phenotypes. Existing datasets suffer from limitations such as small scale, limited species coverage, and inadequate annotation of keypoints for measuring refined and complex morphological phenotypes of fish body parts. To address this gap, we introduce FishPhenoKey, a comprehensive dataset comprising 23,331 high-resolution images spanning six fish species. Notably, FishPhenoKey includes 22 phenotype-oriented annotations, enabling the capture of intricate morphological phenotypes. Motivated by the nuanced evaluation of these subtle morphologies, we also propose a new evaluation metric, Percentage of Measured Phenotype (PMP). It is designed to assess the accuracy of individual keypoint positions and is highly sensitive to the phenotypes measured using the corresponding keypoints. To enhance keypoint detection accuracy, we further propose a novel loss, Anatomically-Calibrated Regularization (ACR), that can be integrated into keypoint detection models, leveraging biological insights to refine keypoint localization. Our contributions set a new benchmark in fish phenotype analysis, addressing the challenges of precise morphological quantification and opening new avenues for research in sustainable aquaculture and genetic studies. Our dataset and code are available at https://github.com/WeizhenLiuBioinform/Fish-Phenotype-Detect.

Figures (7)

• Figure 1: Examples of four fish species with 22 defined keypoints. Each red point represents an anatomical keypoint.
• Figure 2: Visualization of the impact of evaluation metrics on keypoint prediction accuracy. The red dot represents the true position of the keypoint, while the green dot represents the predicted position. The violin charts show the distributions of positional deviations calculated as Euclidean distance (measured as the number of pixels) between predictions and ground truths of keypoints using three evaluation metrics. Notably, PMP demonstrates superiority due to its smallest positional deviations.
• Figure 3: Examples of 22 annotated keypoints and their derived 23 morphological phenotypes on a mottled naked carp fish. Each red point represents a keypoint, and the red numbers 1-22 correspond to individual keypoint classes. Black abbreviations represent the phenotype names, totaling 23 phenotypes, and the blue solid lines indicate the relationships between the phenotypes and keypoints.
• Figure 4: (a) The overall framework for fish keypoint detection integrated with the proposed Anatomically-Calibrated Regularization (ACR) loss. (b) Extraction of fish anatomical structure knowledge from the ground truth keypoints for a given fish species. The distributions are the normalized coordinates of keypoint positions. The normalization process is described in Sec. \ref{['sec:method:acr']}. (c) Conceptualized motivation for ACR loss. The green line segment denotes the "normal" region based on population study and the red one denotes the "abnormal" region. By minimizing the ACR loss, we aim to push the predicted keypoints from "abnormal" region to "normal" region.
• Figure 5: Visualizations of keypoint detection results using OKS/PCK and PMP as evaluation metrics on common carp. The first row shows the results of OKS/PCK and the second row shows the results of PCK. Three keypoints of interest are chosen due to their practical prediction difficulty. Under the same training protocol, the best model weights are selected under three evaluation metrics for prediction on the test images, where OKS and PCK select the same model. In each figure, the green dot is the predicted keypoint and the red dot is the annotated ground truth. PMP shows obvious better performance for these difficult keypoints.