The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition

TL;DR

PanAf-FGBG tackles the challenge of background bias in wildlife-behaviour recognition by releasing a paired foreground-background camera-trap dataset for chimpanzees, enabling direct in-distribution and out-of-distribution evaluation across overlapping and disjoint camera locations. The work demonstrates that background cues are strong predictors of behaviour, introduces a latent-space background neutralisation method that yields notable gains ($+$5.42 ext{ } ext{ extpercent}$ for CNNs and $+$3.75 ext{ } ext{ extpercent}$ for transformers), and systematically analyzes the impact of background duration on learning. It also shows real-world backgrounds outperform synthetic ones in mitigation experiments and provides detailed baselines and configurations to study background effects in wildlife CV. Overall, PanAf-FGBG offers a valuable, realistic benchmark for understanding and improving generalization in wildlife-behaviour recognition with practical implications for conservation analytics.

Abstract

Computer vision analysis of camera trap video footage is essential for wildlife conservation, as captured behaviours offer some of the earliest indicators of changes in population health. Recently, several high-impact animal behaviour datasets and methods have been introduced to encourage their use; however, the role of behaviour-correlated background information and its significant effect on out-of-distribution generalisation remain unexplored. In response, we present the PanAf-FGBG dataset, featuring 20 hours of wild chimpanzee behaviours, recorded at over 350 individual camera locations. Uniquely, it pairs every video with a chimpanzee (referred to as a foreground video) with a corresponding background video (with no chimpanzee) from the same camera location. We present two views of the dataset: one with overlapping camera locations and one with disjoint locations. This setup enables, for the first time, direct evaluation of in-distribution and out-of-distribution conditions, and for the impact of backgrounds on behaviour recognition models to be quantified. All clips come with rich behavioural annotations and metadata including unique camera IDs and detailed textual scene descriptions. Additionally, we establish several baselines and present a highly effective latent-space normalisation technique that boosts out-of-distribution performance by +5.42% mAP for convolutional and +3.75% mAP for transformer-based models. Finally, we provide an in-depth analysis on the role of backgrounds in out-of-distribution behaviour recognition, including the so far unexplored impact of background durations (i.e., the count of background frames within foreground videos).

Figures (13)

• Figure 1: Conceptual Overview. The PanAf-FGBG dataset comprises $>$20 hours of paired and richly annotated foreground-background camera trap videos of wild chimpanzees. The dataset unlocks systematic analyses of the impact of background information on wildlife behaviour recognition. We provide baselines, quantify the background impact on performance, and demonstrate that utilising background information in latent space (see blue data point) can significantly improve recognition performance in this challenging domain.
• Figure 1: Proportion of videos from each country and research site. The inner ring displays the proportion of videos extracted from each country, while the outer ring represents individual research sites. Each research site segment is a unique shade derived from its corresponding country's colour. All proportions are shown in brackets. Note that research site names are replaced with letters to protect the location of the chimps.
• Figure 2: Distribution of Behaviour. Proportion of behaviours in the dataset where smaller segments are colour coordinated.
• Figure 2: Comparison of the proportion of videos containing each behaviour between overlapping and disjoint datasets. Behaviours are ordered from highest to lowest proportion, with exact values displayed above each bar for easy comparison.
• Figure 3: Distribution of Habitat and Location Type. The pie chart gives the proportion of videos taken at each of the three main habitats: forest, marshes and savannah.The histogram shows the total number of videos extracted from each location type.