Benchmarking pig detection and tracking under diverse and challenging conditions

TL;DR

This work delivers two public benchmark datasets, PigDetect for pig detection and PigTrack for pig tracking, gathered from diverse barn environments and annotated with challenging cases to stress modern algorithms. Through systematic benchmarking of state-of-the-art general-purpose detectors and trackers, the study finds that data-centric improvements (e.g., including challenging images) yield notable gains, that SORT-based trackers often outperform end-to-end models on detection while end-to-end models excel at association, and that strong generalization relies on high-quality, diverse training data. Oracle analyses imply detection quality is a key bottleneck for tracking in realistic pens, while third-party transfers demonstrate the value of open data for reproducibility and progress. The datasets and code are released to facilitate ongoing development and fair comparisons in precision livestock farming. Overall, the paper advances reproducibility and provides practical insights into model selection based on application needs and data availability.

Abstract

To ensure animal welfare and effective management in pig farming, monitoring individual behavior is a crucial prerequisite. While monitoring tasks have traditionally been carried out manually, advances in machine learning have made it possible to collect individualized information in an increasingly automated way. Central to these methods is the localization of animals across space (object detection) and time (multi-object tracking). Despite extensive research of these two tasks in pig farming, a systematic benchmarking study has not yet been conducted. In this work, we address this gap by curating two datasets: PigDetect for object detection and PigTrack for multi-object tracking. The datasets are based on diverse image and video material from realistic barn conditions, and include challenging scenarios such as occlusions or bad visibility. For object detection, we show that challenging training images improve detection performance beyond what is achievable with randomly sampled images alone. Comparing different approaches, we found that state-of-the-art models offer substantial improvements in detection quality over real-time alternatives. For multi-object tracking, we observed that SORT-based methods achieve superior detection performance compared to end-to-end trainable models. However, end-to-end models show better association performance, suggesting they could become strong alternatives in the future. We also investigate characteristic failure cases of end-to-end models, providing guidance for future improvements. The detection and tracking models trained on our datasets perform well in unseen pens, suggesting good generalization capabilities. This highlights the importance of high-quality training data. The datasets and research code are made publicly available to facilitate reproducibility, re-use and further development.

Figures (13)

• Figure 1: Distributional properties of ground truth annotations and example images of PigDetect. (a) displays the distribution of the number of pigs per image across all images. The bin width is set to one. (b) shows the distribution of a pig’s maximum intersection over union (IoU) with another pig in the same image across all pigs in all images. The bin width is set to 0.025. (c) shows the distribution of a pig's bounding box size relative to image size across all pigs in all images. The bin width is set to 0.005. For subfigures b and c, large values that occur too rarely to be visible in the histograms were excluded. (d) depicts example images from PigDetect that include challenging conditions such as occlusions and bad visibility. The top row shows images from Inno Pig, Old Breed New House, TiPP, and FriSch (left to right). The bottom row shows images from alameer_automated_2022, bergamini_extracting_2021, and two from psota_multi-pig_2019 (left to right).
• Figure 2: Distributional properties of ground truth annotations and examples of PigTrack. (a) displays the distribution of the number of pigs per video across all videos. The bin width is set to one. (b) shows the distribution of a pig’s maximum IoU with another pig in the same video frame across all pigs in all video frames. The bin width is set to 0.025. (c) shows the distribution of a pig's bounding box size relative to video frame size across all pigs in all video frames. The bin width is set to 0.005. (d) displays the distribution of bounding box shifts of the same individual from one frame to the next across all pigs in all video frames. The bin width is set to 0.01. For subfigures b, c, and d, large values that occur too rarely to be visible in the histograms were excluded. (e-h) show example sequences from PigTrack. The top row displays the first frame of the corresponding video, while the bottom row illustrates the trajectories of individual pigs over time, with each pig represented by a distinct color. Each trajectory starts with a circle and ends with a triangle.
• Figure 3: Visualization of challenging detection scenarios with model predictions. Shown are zoomed-in views of particularly difficult regions in two PigDetect test images. False positives are indicated by red dashed boxes, false negatives by red dotted boxes. All predicted bounding boxes with a score $\geq0.2$ are visualized, since low-confidence detections are frequently utilized in downstream tasks, for example in multi-object tracking zhang2022bytetrackaharon2022bot.
• Figure 4: Experiment on the effect of including challenging images during training. For all experimental conditions, we report the average performance across five runs measured in AP (left) and $\text{AP}_{0.5}$ (right). "All" indicates the performance when adding the challenging images on top of the random images for training. The black dotted line serves as a visual reference for this performance.
• Figure 5: Characteristic failure cases of end-to-end trainable trackers. In both subfigures, the predicted IDs in frame t and frame t+1 are indicated by the color of the bounding boxes. If the predicted ID is the same in both frames, the bounding box has the same color. (a) shows two examples for MOTRv2. The two-colored dotted bounding boxes in frame t represent overlapping duplicate predictions made by the model in case of occlusions. In other words, in these cases the model produces two distinct bounding boxes with different predicted IDs at the same location. In frame t+1, the corresponding two IDs are not predicted at all. (b) shows two examples for MOTIP. The dotted bounding boxes indicate where errors happen. For instance, in the top row, the central pig is incorrectly assigned a different ID in frame t+1 even though it barely moved.