PriVi: Towards A General-Purpose Video Model For Primate Behavior In The Wild

TL;DR

PriVi tackles the limited generalization of primate behavior analysis by introducing a large-scale, primate-centric pretraining dataset (PriVi) that blends Behavior & Research & Observation data with curated YouTube footage. Using a self-supervised V-JEPA backbone trained on PriVi and a lightweight frozen attentive classifier, the approach achieves state-of-the-art performance across four primate behavior datasets with improved data efficiency, especially under low-label regimes. The work further demonstrates that broad, domain-specific pretraining coupled with in-domain continual pretraining (CID) yields consistent gains, suggesting a practical pathway toward unified primate behavior models for cognition, ecology, and conservation research. Overall, PriVi exemplifies how data-centric design and domain alignment can surpass human-centric baselines, enabling scalable, transferable primate behavior understanding with limited labeled data.

Abstract

Non-human primates are our closest living relatives, and analyzing their behavior is central to research in cognition, evolution, and conservation. Computer vision could greatly aid this research, but existing methods often rely on human-centric pretrained models and focus on single datasets, which limits generalization. We address this limitation by shifting from a model-centric to a data-centric approach and introduce PriVi, a large-scale primate-centric video pretraining dataset. PriVi contains 424 hours of curated video, combining 174 hours from behavioral research across 11 settings with 250 hours of diverse web-sourced footage, assembled through a scalable data curation pipeline. We pretrain V-JEPA, a large-scale video model, on PriVi to learn primate-specific representations and evaluate it using a lightweight frozen classifier. Across four benchmark datasets, ChimpACT, BaboonLand, PanAf500, and ChimpBehave, our approach consistently outperforms prior work, including fully finetuned baselines, and scales favorably with fewer labels. These results demonstrate that primate-centric pretraining substantially improves data efficiency and generalization, making it a promising approach for low-label applications. Code, models, and the majority of the dataset will be made available.

Figures (6)

• Figure 1: Continual pretraining on our diverse, large-scale dataset PriVi surpasses state-of-the-art (SOTA) models across four behavior recognition datasets. We recognize behaviors using frozen evaluation. Continual in-domain pretraining (CID) using self-supervised learning (SSL) further improves performance on most datasets. We show relative improvement of mAP (Chimp-ACT) or accuracy (others) compared to SOTA. Images: ours, brookes_panaf20k_2024.
• Figure 2: Our preprocessing pipeline for PriVi. YouTube data is filtered using a learned relevance classifier, while R&O is subsampled based on source dataset metadata. We apply zero-shot primate detection on both to generate bounding boxes and discard empty frames. PriVi consists of 424 h of unique video complete with bounding boxes and CLIP embeddings for keyframes.
• Figure 3: Our pretraining and evaluation architecture. EMA: exponential moving average. $\|$: sequence concatenation. A. We continue self-supervised pretraining of a V-JEPA model on our primate dataset PriVi, doing masked prediction in latent space. B. We train a classifier for each target dataset. Images are ours or brookes_panaf20k_2024.
• Figure 3: Our method outperforms prior methods. We compare our attentive classifier with V-JEPA pretrained on only human-centric data (V-JEPA), pretrained on PriVi, and with continual in-domain pretraining (CID) to various baseline and state-of-the-art methods. $^*$AlphaChimp solves the harder task of simultaneously predicting bounding boxes and cannot be evaluated with ground truth bounding boxes; results are our own reproduction due to incompatible evaluation protocols, see Appendix. Results on test sets.
• Figure 4: Example predictions of our model on PanAf500 and ChimpACT. We show annotated ground truth (gt) and predictions (pred) of the model pretrained on PriVi only. Examples from the validation sets. Image sources: ma_chimpact_2023brookes_panaf20k_2024.