Compact Hypercube Embeddings for Fast Text-based Wildlife Observation Retrieval

TL;DR

The paper addresses scalable text-based retrieval over large multimodal wildlife archives by introducing compact $b$-bit hypercube embeddings that map textual descriptions and observations (images or audio) into a shared Hamming space. Extending the Cross-View Code Alignment framework, it aligns text with visual and acoustic observations using lightweight hashing heads and pretrained wildlife encoders (BioCLIP and BioLingual) with parameter-efficient fine-tuning (LoRA). Key contributions include the first binary, text-based wildlife retrieval framework, a cross-modal hashing objective with anti-collapse regularization, and empirical results showing that 256-bit hashing can match or exceed continuous embeddings while offering substantial memory and speed advantages, along with improved zero-shot generalization under domain shift. The approach enables scalable, interactive exploration of massive wildlife archives and has practical impact for biodiversity monitoring, conservation, and citizen science.

Abstract

Large-scale biodiversity monitoring platforms increasingly rely on multimodal wildlife observations. While recent foundation models enable rich semantic representations across vision, audio, and language, retrieving relevant observations from massive archives remains challenging due to the computational cost of high-dimensional similarity search. In this work, we introduce compact hypercube embeddings for fast text-based wildlife observation retrieval, a framework that enables efficient text-based search over large-scale wildlife image and audio databases using compact binary representations. Building on the cross-view code alignment hashing framework, we extend lightweight hashing beyond a single-modality setup to align natural language descriptions with visual or acoustic observations in a shared Hamming space. Our approach leverages pretrained wildlife foundation models, including BioCLIP and BioLingual, and adapts them efficiently for hashing using parameter-efficient fine-tuning. We evaluate our method on large-scale benchmarks, including iNaturalist2024 for text-to-image retrieval and iNatSounds2024 for text-to-audio retrieval, as well as multiple soundscape datasets to assess robustness under domain shift. Results show that retrieval using discrete hypercube embeddings achieves competitive, and in several cases superior, performance compared to continuous embeddings, while drastically reducing memory and search cost. Moreover, we observe that the hashing objective consistently improves the underlying encoder representations, leading to stronger retrieval and zero-shot generalization. These results demonstrate that binary, language-based retrieval enables scalable and efficient search over large wildlife archives for biodiversity monitoring systems.

Figures (1)

• Figure 1: Overview of the proposed text--observation hashing framework for wildlife retrieval. Textual species descriptions and wildlife observations (images or audio) are encoded into compact binary codes and aligned using cross-view code alignment.