Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SoundCollage: Automated Discovery of New Classes in Audio Datasets

Ryuhaerang Choi, Soumyajit Chatterjee, Dimitris Spathis, Sung-Ju Lee, Fahim Kawsar, Mohammad Malekzadeh

TL;DR

SoundCollage tackles the problem of reusing existing audio datasets by automatically discovering and labeling new classes within the data. It combines a signal pre-processing pipeline that decomposes audio, an unsupervised task-discovery mechanism based on agreement score, and automated labeling via pretrained audio-event classifiers, plus a new semantic clarity measure. Empirical results on AudioSet and a held-out FSD50K show higher class-clarity and improved downstream classifier accuracy, demonstrating enhanced dataset reusability. The approach favors diverse, meaningful new classes over trivial labels like Silence and provides a practical tool for expanding label spaces without additional data collection.

Abstract

Developing new machine learning applications often requires the collection of new datasets. However, existing datasets may already contain relevant information to train models for new purposes. We propose SoundCollage: a framework to discover new classes within audio datasets by incorporating (1) an audio pre-processing pipeline to decompose different sounds in audio samples, and (2) an automated model-based annotation mechanism to identify the discovered classes. Furthermore, we introduce the clarity measure to assess the coherence of the discovered classes for better training new downstream applications. Our evaluations show that the accuracy of downstream audio classifiers within discovered class samples and a held-out dataset improves over the baseline by up to 34.7% and 4.5%, respectively. These results highlight the potential of SoundCollage in making datasets reusable by labeling with newly discovered classes. To encourage further research in this area, we open-source our code at https://github.com/nokia-bell-labs/audio-class-discovery.

SoundCollage: Automated Discovery of New Classes in Audio Datasets

TL;DR

SoundCollage tackles the problem of reusing existing audio datasets by automatically discovering and labeling new classes within the data. It combines a signal pre-processing pipeline that decomposes audio, an unsupervised task-discovery mechanism based on agreement score, and automated labeling via pretrained audio-event classifiers, plus a new semantic clarity measure. Empirical results on AudioSet and a held-out FSD50K show higher class-clarity and improved downstream classifier accuracy, demonstrating enhanced dataset reusability. The approach favors diverse, meaningful new classes over trivial labels like Silence and provides a practical tool for expanding label spaces without additional data collection.

Abstract

Developing new machine learning applications often requires the collection of new datasets. However, existing datasets may already contain relevant information to train models for new purposes. We propose SoundCollage: a framework to discover new classes within audio datasets by incorporating (1) an audio pre-processing pipeline to decompose different sounds in audio samples, and (2) an automated model-based annotation mechanism to identify the discovered classes. Furthermore, we introduce the clarity measure to assess the coherence of the discovered classes for better training new downstream applications. Our evaluations show that the accuracy of downstream audio classifiers within discovered class samples and a held-out dataset improves over the baseline by up to 34.7% and 4.5%, respectively. These results highlight the potential of SoundCollage in making datasets reusable by labeling with newly discovered classes. To encourage further research in this area, we open-source our code at https://github.com/nokia-bell-labs/audio-class-discovery.

Paper Structure

This paper contains 14 sections, 2 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Signal Pre-processing Pipeline
  4. Task Discovery
  5. Automated Labeling of Discovered Classes
  6. Clarity Measure
  7. Evaluation
  8. Datasets
  9. Implementation
  10. Baseline Comparison
  11. Cross-data Quantitative Analysis
  12. Clarity vs Agreement Score
  13. Limitations and Future Work.
  14. Conclusion

Figures (2)

  • Figure 1: Conceptual illustration of SoundCollage.
  • Figure 2: Top-5 discovered classes ranked according to clarity for -- (a) speech, (b) domestic, and (c) natural sounds subsets from AudioSet. Our approach discovers new classes with both higher clarity and agreement.