WildIng: A Wildlife Image Invariant Representation Model for Geographical Domain Shift

TL;DR

Experiments show that WildIng enhances the accuracy of foundation models such as BioCLIP by 30% under geographical domain shift conditions and integrates text descriptions with image features, creating a more robust representation to geographical domain shifts.

Abstract

Wildlife monitoring is crucial for studying biodiversity loss and climate change. Camera trap images provide a non-intrusive method for analyzing animal populations and identifying ecological patterns over time. However, manual analysis is time-consuming and resource-intensive. Deep learning, particularly foundation models, has been applied to automate wildlife identification, achieving strong performance when tested on data from the same geographical locations as their training sets. Yet, despite their promise, these models struggle to generalize to new geographical areas, leading to significant performance drops. For example, training an advanced vision-language model, such as CLIP with an adapter, on an African dataset achieves an accuracy of 84.77%. However, this performance drops significantly to 16.17% when the model is tested on an American dataset. This limitation partly arises because existing models rely predominantly on image-based representations, making them sensitive to geographical data distribution shifts, such as variation in background, lighting, and environmental conditions. To address this, we introduce WildIng, a Wildlife image Invariant representation model for geographical domain shift. WildIng integrates text descriptions with image features, creating a more robust representation to geographical domain shifts. By leveraging textual descriptions, our approach captures consistent semantic information, such as detailed descriptions of the appearance of the species, improving generalization across different geographical locations. Experiments show that WildIng enhances the accuracy of foundation models such as BioCLIP by 30% under geographical domain shift conditions. We evaluate WildIng on two datasets collected from different regions, namely America and Africa. The code and models are publicly available at https://github.com/Julian075/CATALOG/tree/WildIng.

Figures (10)

• Figure 1: Comparison of WildIng and WildCLIP gabeff2024wildclip under geographical domain shift. Both models are trained on the Snapshot Serengeti dataset from Africa dryad_5pt92 and evaluated on the Terra Incognita dataset from the United States beery2018recognition. WildIng demonstrates superior performance.
• Figure 2: Overview of WildIng. The model integrates image, text, and image-text encoders along with an LLM. By leveraging text descriptions and image features, it extracts invariant features, improving robustness against geographical domain shifts.
• Figure 3: Detailed illustration of the image-text module, which consists of a VLM, a text encoder, and an MLP. This module processes input images and converts them into image-text embeddings.
• Figure 4: Cropped images from the Snapshot Serengeti and Terra Incognita datasets where we observe the geographical domain shift and the difference in classes (different taxonomic groups).
• Figure 5: Class distribution of the Serengeti dataset.