Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CattleFace-RGBT: RGB-T Cattle Facial Landmark Benchmark

Ethan Coffman, Reagan Clark, Nhat-Tan Bui, Trong Thang Pham, Beth Kegley, Jeremy G. Powell, Jiangchao Zhao, Ngan Le

TL;DR

The paper tackles the need for temperature-aware cattle welfare assessment by introducing CattleFace-RGBT, the first RGB-T cattle facial landmark dataset with 2,300 RGB-thermal pairs and 13 keypoints. It employs a semi-automatic annotation pipeline that transfers RGB-trained models to the thermal domain and iteratively refines annotations, facilitated by a custom C# annotation tool. The dataset and benchmarks span multiple backbones (e.g., ResNet, ViT, Swin) using Detectron2, establishing baselines for bounding-box and keypoint detection in both RGB and thermal modalities. This resource enables temperature-based welfare monitoring and fever detection in farm settings, with code and data available at the project repository for broader adoption.

Abstract

To address this challenge, we introduce CattleFace-RGBT, a RGB-T Cattle Facial Landmark dataset consisting of 2,300 RGB-T image pairs, a total of 4,600 images. Creating a landmark dataset is time-consuming, but AI-assisted annotation can help. However, applying AI to thermal images is challenging due to suboptimal results from direct thermal training and infeasible RGB-thermal alignment due to different camera views. Therefore, we opt to transfer models trained on RGB to thermal images and refine them using our AI-assisted annotation tool following a semi-automatic annotation approach. Accurately localizing facial key points on both RGB and thermal images enables us to not only discern the cattle's respiratory signs but also measure temperatures to assess the animal's thermal state. To the best of our knowledge, this is the first dataset for the cattle facial landmark on RGB-T images. We conduct benchmarking of the CattleFace-RGBT dataset across various backbone architectures, with the objective of establishing baselines for future research, analysis, and comparison. The dataset and models are at https://github.com/UARK-AICV/CattleFace-RGBT-benchmark

CattleFace-RGBT: RGB-T Cattle Facial Landmark Benchmark

TL;DR

The paper tackles the need for temperature-aware cattle welfare assessment by introducing CattleFace-RGBT, the first RGB-T cattle facial landmark dataset with 2,300 RGB-thermal pairs and 13 keypoints. It employs a semi-automatic annotation pipeline that transfers RGB-trained models to the thermal domain and iteratively refines annotations, facilitated by a custom C# annotation tool. The dataset and benchmarks span multiple backbones (e.g., ResNet, ViT, Swin) using Detectron2, establishing baselines for bounding-box and keypoint detection in both RGB and thermal modalities. This resource enables temperature-based welfare monitoring and fever detection in farm settings, with code and data available at the project repository for broader adoption.

Abstract

To address this challenge, we introduce CattleFace-RGBT, a RGB-T Cattle Facial Landmark dataset consisting of 2,300 RGB-T image pairs, a total of 4,600 images. Creating a landmark dataset is time-consuming, but AI-assisted annotation can help. However, applying AI to thermal images is challenging due to suboptimal results from direct thermal training and infeasible RGB-thermal alignment due to different camera views. Therefore, we opt to transfer models trained on RGB to thermal images and refine them using our AI-assisted annotation tool following a semi-automatic annotation approach. Accurately localizing facial key points on both RGB and thermal images enables us to not only discern the cattle's respiratory signs but also measure temperatures to assess the animal's thermal state. To the best of our knowledge, this is the first dataset for the cattle facial landmark on RGB-T images. We conduct benchmarking of the CattleFace-RGBT dataset across various backbone architectures, with the objective of establishing baselines for future research, analysis, and comparison. The dataset and models are at https://github.com/UARK-AICV/CattleFace-RGBT-benchmark
Paper Structure (9 sections, 5 figures, 2 tables)

This paper contains 9 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Datasets
  3. The CattleFace-RGBT Dataset
  4. Image Collection and Preprocessing
  5. AI-assisted Annotation
  6. The Annotation Tool
  7. Landmarks
  8. Benchmarking CattleFace-RGBT Dataset
  9. Conclusion

Figures (5)

  • Figure 1: Example of RGB and thermal pair image with 13 annotated keypoints annotated in our CattleFace-RGBT dataset.
  • Figure 2: Examples of undesired images.
  • Figure 3: The workflow for creating our dataset.
  • Figure 4: The user interface of our annotation tool with the utilities on the right including "Browse","Save", "Reload", "Next", "Prev", "Discard".
  • Figure 5: Illustration of 13 keypoint landmarks.