BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity

TL;DR

BIOSCAN-5M tackles the need for large-scale, multi-modal biodiversity data by providing over 5 million arthropod specimens annotated with images, DNA barcodes, BINs, taxonomy, geography, and size. The authors develop and evaluate three benchmarks—DNA-based taxonomic classification via semi-supervised pretraining, zero-shot clustering across modalities, and multimodal retrieval with cross-modal embedding alignment—demonstrating substantial gains from modality integration and dataset scale. The work highlights resourceful data curation, a robust partitioning scheme for closed- and open-world tasks, and state-of-the-art performance on several objectives, underscoring the practical impact for biodiversity monitoring and ML research. The release of BIOSCAN-5M and accompanying code fosters reproducibility and enables future work that can integrate geographic and size data with multi-modal models for even more accurate and robust species discovery and identification.

Abstract

As part of an ongoing worldwide effort to comprehend and monitor insect biodiversity, this paper presents the BIOSCAN-5M Insect dataset to the machine learning community and establish several benchmark tasks. BIOSCAN-5M is a comprehensive dataset containing multi-modal information for over 5 million insect specimens, and it significantly expands existing image-based biological datasets by including taxonomic labels, raw nucleotide barcode sequences, assigned barcode index numbers, geographical, and size information. We propose three benchmark experiments to demonstrate the impact of the multi-modal data types on the classification and clustering accuracy. First, we pretrain a masked language model on the DNA barcode sequences of the BIOSCAN-5M dataset, and demonstrate the impact of using this large reference library on species- and genus-level classification performance. Second, we propose a zero-shot transfer learning task applied to images and DNA barcodes to cluster feature embeddings obtained from self-supervised learning, to investigate whether meaningful clusters can be derived from these representation embeddings. Third, we benchmark multi-modality by performing contrastive learning on DNA barcodes, image data, and taxonomic information. This yields a general shared embedding space enabling taxonomic classification using multiple types of information and modalities. The code repository of the BIOSCAN-5M Insect dataset is available at https://github.com/bioscan-ml/BIOSCAN-5M.

Figures (25)

• Figure 1: Record attributes. The BIOSCAN-5M dataset provides taxonomic labels, a DNA barcode sequence, barcode index number, a high-resolution image along with its cropped and resized versions, as well as size and geographic information for each sample.
• Figure 2: Samples of original full-size images of distinct organisms in the BIOSCAN-5M dataset.
• Figure 3: Geographical locations obtained from latitude and longitude coordinates of the regions where the samples of the BIOSCAN-5M dataset were collected.
• Figure 4: Zero-shot clustering AMI (%) performance across taxonomic ranks. Left: Image encoders. Right: DNA encoders.
• Figure 5: DNA-based taxonomic classification methodology. Two stages of the proposed semi-supervised learning set-up based on BarcodeBERT millan2023barcodebert. (1) Pretraining: DNA sequences are tokenized using non-overlapping $k$-mers and 50% of the tokens are masked for the MLM task. Tokens are encoded and fed into a transformer model. The output embeddings are used for token-level classification. (2) Fine-tuning: All DNA sequences in a dataset are tokenized using non-overlapping $k$-mer tokenization and all tokenized sequences, without masking, are passed through the pretrained transformer model. Global mean-pooling is applied over the token-level embeddings and the output is used for taxonomic classification.