WildLive: Near Real-time Visual Wildlife Tracking onboard UAVs

TL;DR

WildLive tackles onboard, near real-time wildlife tracking directly from UAV video by fusing SAHI-boosted YOLO detection/segmentation with sparse Lucas-Kanade tracking. The approach focuses computation on high-uncertainty regions, achieving up to 17.81 fps on HD and 7.53 fps on 4K on Jetson Orin AGX, while maintaining strong MOT accuracy (MOTA 81.17%, IDF1 79.03%) and enabling robust tracklet management via PoA/IoU and a confidence accumulator. A publicly released WildLive Benchmark Dataset (over 215k bounding boxes, 291 tracklets across 22 four-kay UAV sequences) supports reproducibility and benchmarking against ByteTrack, OCSORT, and SORT. The work demonstrates the practicality of onboard, autonomous wildlife monitoring with potential BVLOS navigation applications, and outlines future enhancements for small-object detection and broader autonomous integration.

Abstract

Live tracking of wildlife via high-resolution video processing directly onboard drones is widely unexplored and most existing solutions rely on streaming video to ground stations to support navigation. Yet, both autonomous animal-reactive flight control beyond visual line of sight and/or mission-specific individual and behaviour recognition tasks rely to some degree on this capability. In response, we introduce WildLive - a near real-time animal detection and tracking framework for high-resolution imagery running directly onboard uncrewed aerial vehicles (UAVs). The system performs multi-animal detection and tracking at 17.81fps for HD and 7.53fps on 4K video streams suitable for operation during higher altitude flights to minimise animal disturbance. Our system is optimised for Jetson Orin AGX onboard hardware. It integrates the efficiency of sparse optical flow tracking and mission-specific sampling with device-optimised and proven YOLO-driven object detection and segmentation techniques. Essentially, computational resource is focused onto spatio-temporal regions of high uncertainty to significantly improve UAV processing speeds. Alongside, we introduce our WildLive dataset, which comprises 200K+ annotated animal instances across 19K+ frames from 4K UAV videos collected at the Ol Pejeta Conservancy in Kenya. All frames contain ground truth bounding boxes, segmentation masks, as well as individual tracklets and tracking point trajectories. We compare our system against current object tracking approaches including OC-SORT, ByteTrack, and SORT. Our multi-animal tracking experiments with onboard hardware confirm that near real-time high-resolution wildlife tracking is possible on UAVs whilst maintaining high accuracy levels as needed for future navigational and mission-specific animal-centric operational autonomy. Our materials are available at: https://dat-nguyenvn.github.io/WildLive/

Figures (3)

• Figure 1: WildLive System Overview. Our pioneering approach integrates the efficiency of (a) Slicing-aided Hyper Inference with proven YOLO-driven (b) object re-detection and (c) segmentation techniques. The framework exploits animal-associated (d) inexpensive Lucas-Kanade point tracks to interpolate intermittent re-detection allowing (e) high-speed HD/4K tracking directly onboard UAVs utilising (f) custom drones with Jetson Orin AGX hardware.
• Figure 2: WildLive Benchmark Dataset Overview. 19 representative 4K frames (each sampled from a different video) showcasing the dataset's diversity regarding altitudes, environments, species, approach angles as well as view points. The top right image shows a zoomed-in example patch with ground truth annotations of animal bounding boxes, segmentations, and tracked point trajectories.
• Figure 3: Distribution of Animal Resolutions. We show the width and height distributions of ground truth animal bounding boxes in the WildLive Benchmark dataset together with a typical animal patch (sampled from the peak). Distributions peak at about 100 pixels and tail off rapidly (note logarithmic plot scales).