Learning to detect an animal sound from five examples

TL;DR

The paper reframes bioacoustic sound event detection as a few-shot learning problem to cope with the scarcity of strongly-labeled data across diverse taxa. It assembles 14 open datasets and runs a public challenge to evaluate few-shot SED methods, showing that prototypical networks with task-specific adaptations perform strongly, especially when accounting for event duration and non-stationarity. Through extensive analyses, it reveals that dataset characteristics alone do not predict performance, highlights the value of test-time adaptation and smart negative exemplars, and demonstrates that expert users view such systems as useful aids for manual annotation. The work argues for a generalizable, post-template M-L approach to SED that can operate across many biological domains, potentially reducing labeling costs and enabling scalable biodiversity monitoring. It also points to the potential of a single robust embedding to underpin cross-dataset bioacoustic SED, motivating future benchmarks and system designs.

Abstract

Automatic detection and classification of animal sounds has many applications in biodiversity monitoring and animal behaviour. In the past twenty years, the volume of digitised wildlife sound available has massively increased, and automatic classification through deep learning now shows strong results. However, bioacoustics is not a single task but a vast range of small-scale tasks (such as individual ID, call type, emotional indication) with wide variety in data characteristics, and most bioacoustic tasks do not come with strongly-labelled training data. The standard paradigm of supervised learning, focussed on a single large-scale dataset and/or a generic pre-trained algorithm, is insufficient. In this work we recast bioacoustic sound event detection within the AI framework of few-shot learning. We adapt this framework to sound event detection, such that a system can be given the annotated start/end times of as few as 5 events, and can then detect events in long-duration audio -- even when the sound category was not known at the time of algorithm training. We introduce a collection of open datasets designed to strongly test a system's ability to perform few-shot sound event detections, and we present the results of a public contest to address the task. We show that prototypical networks are a strong-performing method, when enhanced with adaptations for general characteristics of animal sounds. We demonstrate that widely-varying sound event durations are an important factor in performance, as well as non-stationarity, i.e. gradual changes in conditions throughout the duration of a recording. For fine-grained bioacoustic recognition tasks without massive annotated training data, our results demonstrate that few-shot sound event detection is a powerful new method, strongly outperforming traditional signal-processing detection methods in the fully automated scenario.

Figures (8)

• Figure 1: (a) Few-shot sound event detection: the first 5 sound events are given as examples---in standard supervised learning they would be considered the training set---and the remainder must then be detected. (b) Few-shot sound event detection as a meta-learning problem. Each of our datasets represents a different but related few-shot task. The overall goal is to use the training and validation datasets collectively to train or otherwise develop a system that, when presented with 5 sound events from any of the evaluation datasets, can perform well at detecting the remaining events.
• Figure 2: Sample spectrograms for each dataset. POS (positive, i.e. target) vocalizations are indicated with a white rectangle.
• Figure 3: Spectral summary profiles of each dataset. For each frequency, we show mean and 90% confidence intervals of the energy distribution, for the foreground (POS events) and negative regions (background and non target sounds) separately.
• Figure 4: Temporal profiles of each dataset. We show the empirical distributions (kde smoothed) of durations of marked regions, for the foreground (POS events) and negative regions (all non-POS regions) separately.
• Figure 5: Values of similarity between the annotated calls and the first 5 events (shots), and stereotypy for each class in the evaluation set. Classes are indicated in the horizontal axis by DatasetName_ClassName. The similarity metric is based on the average maximum cross correlation between events. It ranges between 0 and 1, where values closer to 1 represent higher similarity. (details in \ref{['app:sterotypy']}).