Can 3D point cloud data improve automated body condition score prediction in dairy cattle?

TL;DR

Three-dimensional point clouds do not provide a consistent advantage over depth images for BCS prediction in dairy cattle under the evaluated conditions, indicating that direct head-to-head comparisons with depth image-based approaches remain limited.

Abstract

Body condition score (BCS) is a widely used indicator of body energy status and is closely associated with metabolic status, reproductive performance, and health in dairy cattle; however, conventional visual scoring is subjective and labor-intensive. Computer vision approaches have been applied to BCS prediction, with depth images widely used because they capture geometric information independent of coat color and texture. More recently, three-dimensional point cloud data have attracted increasing interest due to their ability to represent richer geometric characteristics of animal morphology, but direct head-to-head comparisons with depth image-based approaches remain limited. In this study, we compared top-view depth image and point cloud data for BCS prediction under four settings: 1) unsegmented raw data, 2) segmented full-body data, 3) segmented hindquarter data, and 4) handcrafted feature data. Prediction models were evaluated using data from 1,020 dairy cows collected on a commercial farm, with cow-level cross-validation to prevent data leakage. Depth image-based models consistently achieved higher accuracy than point cloud-based models when unsegmented raw data and segmented full-body data were used, whereas comparable performance was observed when segmented hindquarter data were used. Both depth image and point cloud approaches showed reduced accuracy when handcrafted feature data were employed compared with the other settings. Overall, point cloud-based predictions were more sensitive to noise and model architecture than depth image-based predictions. Taken together, these results indicate that three-dimensional point clouds do not provide a consistent advantage over depth images for BCS prediction in dairy cattle under the evaluated conditions.

Figures (4)

• Figure 1: Overview of the four experimental settings used in this study: 1) unsegmented raw depth data, 2) segmented full-body data, 3) segmented hindquarter data, and 4) handcrafted feature data. In setting 1, raw depth CSV files were converted into depth images and three-dimensional point clouds. RGB images were used for cow segmentation and keypoint detection, providing region-of-interest masks and anatomical landmarks for settings 2 and 3. The detected keypoints were further used in setting 4 to extract handcrafted geometric features, including distance and area features derived from two-point landmark pairs, as well as volumetric features derived from three-point landmark combinations. The depth image–based deep learning models included ResNet-18 and ConvNeXt, while the point cloud–based deep learning models included PointNet and DGCNN. The machine learning models applied to handcrafted features were random forest and LightGBM.
• Figure 2: Illustration of handcrafted features extracted from depth images. A) The maximum distance and area were calculated from 10 lines (L1, L2, L3, L4, L5, L6, L7, L8, L9, and L10) derived by connecting the nine anatomical keypoints. Using L2 as an example, the maximum distance was defined as the height of the largest bulge of the body surface along L2, while the area was defined as the overall magnitude of body surface bulging along L2. B) The volume was separated into four volumes (V1, V2, V3, and V4). Using V1 as an example, it was defined as the volume between the cow body surface and the triangular anatomical plane defined by Spike A, Spike B, and the right hook point.
• Figure 3: Illustration of handcrafted features extracted from point clouds. A) The maximum distance and area were calculated from 10 lines (L1, L2, L3, L4, L5, L6, L7, L8, L9, and L10) derived by connecting the nine anatomical keypoints. Using L2 as an example, the maximum distance was defined as the height of the largest bulge of the body surface along L2, while the area was defined as the overall magnitude of body surface bulging along L2. B) The volume was separated into four volumes (V1, V2, V3, and V4). Using V1 as an example, it was defined as the volume between the cow body surface and the triangular anatomical plane defined by Spike A, Spike B, and the right hook point.
• Figure 4: Boxplots of cattle-level prediction accuracy across four experimental settings under different error tolerance thresholds. Setting 1 corresponded to unsegmented raw data, setting 2 to segmented full-body data, setting 3 to segmented hindquarter data, and setting 4 to handcrafted feature data. The accuracy on the y-axis represents the proportion of cows that were correctly classified across different body condition score levels, averaged over five replicates. Box colors represent different model architectures, while hatch patterns indicate input modality: solid boxes correspond to depth image inputs, and diagonally hatched boxes correspond to point cloud inputs.