BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics

TL;DR

BirdSet tackles the scarcity of large-scale, domain-specific datasets for audio classification by introducing BirdSet, a large avian bioacoustics benchmark with ≈6,800 training hours spanning ≈10,000 species and eight strongly labeled test datasets. The paper presents three supervised training protocols (lt, mt, dt) and evaluates five model families (including EfficientNet, ConvNext, AST, EAT, and W2V2) under covariate shift, label uncertainty, and task shift, using a standardized, threshold-free evaluation suite that includes $cmAP$, $AUROC$, and $T1\text{-}Acc$, with POW for validation. Key contributions include a unified, publicly accessible dataset collection on Hugging Face, a comprehensive codebase, and an extensive benchmark demonstrating that large-scale pretraining improves generalization across diverse soundscapes, while highlighting the need for domain-specific advances in robust, multi-label environmental audio classification. The work aims to standardize evaluation and foster reproducibility, enabling robust comparisons and enabling future study in self-supervised representation learning within avian bioacoustics and beyond.

Abstract

Deep learning (DL) has greatly advanced audio classification, yet the field is limited by the scarcity of large-scale benchmark datasets that have propelled progress in other domains. While AudioSet is a pivotal step to bridge this gap as a universal-domain dataset, its restricted accessibility and limited range of evaluation use cases challenge its role as the sole resource. Therefore, we introduce BirdSet, a large-scale benchmark dataset for audio classification focusing on avian bioacoustics. BirdSet surpasses AudioSet with over 6,800 recording hours ($\uparrow\!17\%$) from nearly 10,000 classes ($\uparrow\!18\times$) for training and more than 400 hours ($\uparrow\!7\times$) across eight strongly labeled evaluation datasets. It serves as a versatile resource for use cases such as multi-label classification, covariate shift or self-supervised learning. We benchmark six well-known DL models in multi-label classification across three distinct training scenarios and outline further evaluation use cases in audio classification. We host our dataset on Hugging Face for easy accessibility and offer an extensive codebase to reproduce our results.

Figures (15)

• Figure 1: BirdSet's volume compared to broader audio classification datasets. The area of the circles represents the total recording duration in [h]. More details can be found in \ref{['app:birdsetdata']}.
• Figure 2: Typical model training for classification in avian bioacoustics.
• Figure 3: Key obstacles for model robustness in bird sound classification.
• Figure 4: An illustration of segment- and event-based evaluations vanmerrienbour2024.
• Figure 5: BirdSet's geographical distribution of focals for training and soundscapes for testing.