Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Context-Aware Outlier Rejection for Robust Multi-View 3D Tracking of Similar Small Birds in An Outdoor Aviary

Keon Moradi, Ethan Haque, Jasmeen Kaur, Alexandra B. Bentz, Eli S. Bridge, Golnaz Habibi

TL;DR

The paper tackles robust 3D tracking of visually similar small birds in outdoor environments using a five-camera multi-view setup. It introduces a context-aware landmark-based outlier rejection built on Voronoi diagrams of environmental landmarks to improve feature matching and triangulation for 3D reconstruction, enabling reliable multi-bird tracking under occlusions. The method achieves a reported $97\%$ matching accuracy and a $20\%$ outlier reduction, demonstrates strong short-term tracking despite challenging conditions, and provides a large annotated dataset of $80$ birds across five camera views for further research. Overall, the work advances outdoor animal tracking by integrating environmental context into feature matching, yielding improved 3D modeling and behavior-rich spatio-temporal analytics with practical implications for ecology and ethology.

Abstract

This paper presents a novel approach for robust 3D tracking of multiple birds in an outdoor aviary using a multi-camera system. Our method addresses the challenges of visually similar birds and their rapid movements by leveraging environmental landmarks for enhanced feature matching and 3D reconstruction. In our approach, outliers are rejected based on their nearest landmark. This enables precise 3D-modeling and simultaneous tracking of multiple birds. By utilizing environmental context, our approach significantly improves the differentiation between visually similar birds, a key obstacle in existing tracking systems. Experimental results demonstrate the effectiveness of our method, showing a $20\%$ elimination of outliers in the 3D reconstruction process, with a $97\%$ accuracy in matching. This remarkable accuracy in 3D modeling translates to robust and reliable tracking of multiple birds, even in challenging outdoor conditions. Our work not only advances the field of computer vision but also provides a valuable tool for studying bird behavior and movement patterns in natural settings. We also provide a large annotated dataset of 80 birds residing in four enclosures for 20 hours of footage which provides a rich testbed for researchers in computer vision, ornithologists, and ecologists. Code and the link to the dataset is available at https://github.com/airou-lab/3D_Multi_Bird_Tracking

Context-Aware Outlier Rejection for Robust Multi-View 3D Tracking of Similar Small Birds in An Outdoor Aviary

TL;DR

The paper tackles robust 3D tracking of visually similar small birds in outdoor environments using a five-camera multi-view setup. It introduces a context-aware landmark-based outlier rejection built on Voronoi diagrams of environmental landmarks to improve feature matching and triangulation for 3D reconstruction, enabling reliable multi-bird tracking under occlusions. The method achieves a reported matching accuracy and a outlier reduction, demonstrates strong short-term tracking despite challenging conditions, and provides a large annotated dataset of birds across five camera views for further research. Overall, the work advances outdoor animal tracking by integrating environmental context into feature matching, yielding improved 3D modeling and behavior-rich spatio-temporal analytics with practical implications for ecology and ethology.

Abstract

This paper presents a novel approach for robust 3D tracking of multiple birds in an outdoor aviary using a multi-camera system. Our method addresses the challenges of visually similar birds and their rapid movements by leveraging environmental landmarks for enhanced feature matching and 3D reconstruction. In our approach, outliers are rejected based on their nearest landmark. This enables precise 3D-modeling and simultaneous tracking of multiple birds. By utilizing environmental context, our approach significantly improves the differentiation between visually similar birds, a key obstacle in existing tracking systems. Experimental results demonstrate the effectiveness of our method, showing a elimination of outliers in the 3D reconstruction process, with a accuracy in matching. This remarkable accuracy in 3D modeling translates to robust and reliable tracking of multiple birds, even in challenging outdoor conditions. Our work not only advances the field of computer vision but also provides a valuable tool for studying bird behavior and movement patterns in natural settings. We also provide a large annotated dataset of 80 birds residing in four enclosures for 20 hours of footage which provides a rich testbed for researchers in computer vision, ornithologists, and ecologists. Code and the link to the dataset is available at https://github.com/airou-lab/3D_Multi_Bird_Tracking

Paper Structure

This paper contains 18 sections, 4 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Main Contribution
  3. Related Work
  4. Dataset
  5. Multi-View Multi-Bird Tracking Overview
  6. Object Detection
  7. Feature Extraction and Matching
  8. Landmark based Outlier rejection
  9. Voronoi cells with bounded edges
  10. 3D multi-object tracking of the birds
  11. Camera Calibration
  12. 3D reconstruction from multiple camera views
  13. Multi-bird Tracking using Kalman Filter
  14. Experiments
  15. Keypoints Statistics and Outlier Rejection
  16. ...and 3 more sections

Figures (7)

  • Figure 1: Five camera views in one of the enclosures in the aviary, which is studied in this project.
  • Figure 2: Proposed Workflow diagram: our multi-view 3D multi-bird tracking
  • Figure 3: Masking Step. We consider masking algorithms, with the result shown in the bottom row.
  • Figure 4: (a) Keypoints in the first camera view; (b) Keypoints in the second camera view
  • Figure 5: Locally constructed Voronoi diagram.
  • ...and 2 more figures