CrypticBio: A Large Multimodal Dataset for Visually Confusing Biodiversity

TL;DR

CrypticBio tackles the challenge of visually cryptic biodiversity by introducing a massively multimodal dataset that pairs images with rich contextual metadata, including taxonomy, geography, date, and multilingual vernaculars. The approach leverages misidentification patterns to form data-driven cryptic groups and provides an open-source curation pipeline (CrypticBio-Curate) to enable scalable subset creation and benchmarking. Benchmark results with CLIP-style biodiversity models show that incorporating geographic context substantially improves zero-shot performance on cryptic species, underscoring the value of multimodal and context-aware representations. Overall, CrypticBio aims to accelerate trustworthy, real-world biodiversity AI by offering scale, taxonomy breadth, and multimodal context for robust cryptic-species identification and conservation research.

Abstract

We present CrypticBio, the largest publicly available multimodal dataset of visually confusing species, specifically curated to support the development of AI models in the context of biodiversity applications. Visually confusing or cryptic species are groups of two or more taxa that are nearly indistinguishable based on visual characteristics alone. While much existing work addresses taxonomic identification in a broad sense, datasets that directly address the morphological confusion of cryptic species are small, manually curated, and target only a single taxon. Thus, the challenge of identifying such subtle differences in a wide range of taxa remains unaddressed. Curated from real-world trends in species misidentification among community annotators of iNaturalist, CrypticBio contains 52K unique cryptic groups spanning 67K species, represented in 166 million images. Rich research-grade image annotations--including scientific, multicultural, and multilingual species terminology, hierarchical taxonomy, spatiotemporal context, and associated cryptic groups--address multimodal AI in biodiversity research. For easy dataset curation, we provide an open-source pipeline CrypticBio-Curate. The multimodal nature of the dataset beyond vision-language arises from the integration of geographical and temporal data as complementary cues to identifying cryptic species. To highlight the importance of the dataset, we benchmark a suite of state-of-the-art foundation models across CrypticBio subsets of common, unseen, endangered, and invasive species, and demonstrate the substantial impact of geographical context on vision-language zero-shot learning for cryptic species. By introducing CrypticBio, we aim to catalyze progress toward real-world-ready biodiversity AI models capable of handling the nuanced challenges of species ambiguity.

Figures (20)

• Figure 1: Challenges of biodiversity: (1) viewpoint variations (Parasteatoda tepidariorum); (2) occlusion by other objects (Vipera berus); (3) clutter (Harmonia axyridis); (4) multiple life cycle stages (Papilio machaon); (5) deformations (Cornu aspersum); (6) intra-class variation (Passer domesticus); (7) inter-class similarity (Bellis perennis, Leucanthemum vulgare, Chamomile matricaria).
• Figure 2: Example of cryptic species in CrypticBio. Each column shows from left to right cryptic groups from Arachnida, Aves, Insecta, Plantae, Fungi, Mollusca, and Reptilia, taxa representative in biodiversity conservation and policy change supervision.
• Figure 3: Cryptic group size distribution in CrypticBio. The long-tailed distribution suggests that the majority are divided into a small number of cryptic entities.
• Figure 4: Spatiotemporal distribution of CrypticBio: (top) stacked seasonality distribution; (bottom) geographical distribution. Majority of records are concentrated in Europe and North America, with a seasonal peak in observations during May.
• Figure 5: The importance of geospatial information demonstrated by two visually similar species and their distinct patterns in geospatial locations from CrypticBio.