ARIONet: An Advanced Self-supervised Contrastive Representation Network for Birdsong Classification and Future Frame Prediction

TL;DR

ARIONet advances birdsong analysis by unifying self-supervised contrastive learning with future-frame prediction to capture both species-specific acoustic signatures and temporal dynamics. It processes chromagram-based multiview inputs alongside fused spectral features through a transformer encoder, optimizing a dual objective $\mathcal{L}_{\text{con}}$ and $\mathcal{L}_{\text{pred}}$ to learn robust, temporally aware embeddings without labeled data. Empirical results on four diverse Xeno-Canto-derived datasets demonstrate state-of-the-art classification accuracy (e.g., 98.41% on XC-British) and high fidelity in future-frame prediction (cosine similarity up to 0.952), with ablations confirming the necessity of both objectives and domain-specific augmentations. The approach holds promise for scalable, annotation-efficient ecological monitoring and behavioral analysis, while acknowledging ethical considerations and preprocessing limitations such as energy-based frame filtering. Overall, ARIONet offers a robust framework for real-world biodiversity surveillance that leverages temporal structure and pitch dynamics in birdsong.

Abstract

Automated birdsong classification is essential for advancing ecological monitoring and biodiversity studies. Despite recent progress, existing methods often depend heavily on labeled data, use limited feature representations, and overlook temporal dynamics essential for accurate species identification. In this work, we propose a self-supervised contrastive network, ARIONet (Acoustic Representation for Interframe Objective Network), that jointly optimizes contrastive classification and future frame prediction using augmented audio representations. The model simultaneously integrates multiple complementary audio features within a transformer-based encoder model. Our framework is designed with two key objectives: (1) to learn discriminative species-specific representations for contrastive learning through maximizing similarity between augmented views of the same audio segment while pushing apart different samples, and (2) to model temporal dynamics by predicting future audio frames, both without requiring large-scale annotations. We validate our framework on four diverse birdsong datasets, including the British Birdsong Dataset, Bird Song Dataset, and two extended Xeno-Canto subsets (A-M and N-Z). Our method consistently outperforms existing baselines and achieves classification accuracies of 98.41%, 93.07%, 91.89%, and 91.58%, and F1-scores of 97.84%, 94.10%, 91.29%, and 90.94%, respectively. Furthermore, it demonstrates low mean absolute errors and high cosine similarity, up to 95\%, in future frame prediction tasks. Extensive experiments further confirm the effectiveness of our self-supervised learning strategy in capturing complex acoustic patterns and temporal dependencies, as well as its potential for real-world applicability in ecological conservation and monitoring.

Figures (7)

• Figure 1: Overview of the proposed framework. Processed samples are segmented and converted into 4 feature types: temporal, spectral, MFCC, and chromagram. Augmented views are created using pitch shifting, time masking, and chromagram masking, then encoded via a shared transformer with positional embeddings and multi-head attention. Then the projected embeddings are optimized using contrastive and temporal prediction losses.
• Figure 2: Visualization of an original audio sample along with the results of 5% low-energy filtering. (a) shows the original waveform. (c) displays the original Mel-Spectrogram. The resulting (d) Energy Mask identifies frames to keep (1) or remove (0) based on the 5% threshold of the peak frame energy. The resulting masked waveform, containing only the high-energy segments, is shown in (b).
• Figure 3: The sequence of processing and feature extraction is shown by: (a) the original full waveform (before filtering), which is transformed into (b) the energy-filtered full waveform. The corresponding time-frequency visualizations are (c) the Mel Spectrogram (original audio) and (d) the Mel Spectrogram (energy-filtered signal). Finally, the extracted features from the filtered signal include harmonic content with chroma features (e), and (f) mel-frequency cepstral coefficients and the spectral centroid (energy-filtered signal).
• Figure 4: Contrastive learning module: embeddings are projected and normalized, then compared using cosine similarity to form a similarity matrix. The temperature-scaled loss, $\ell'_i$, pulls positive pairs together and pushes negative pairs apart for discriminative representations.
• Figure 5: Embeddings were clustered following contrastive learning training. Embeddings of five species (a) and their corresponding clusters (b) are shown for the XC-BS5 dataset, along with feature embeddings of 85 species (c) and their corresponding clusters (d) identified in the XC-British dataset.