Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Microphone Conversion: Mitigating Device Variability in Sound Event Classification

Myeonghoon Ryu, Hongseok Oh, Suji Lee, Han Park

TL;DR

This work tackles the challenge of device variability in sound event classification (SEC) by introducing Microphone Conversion, a CycleGAN-based data augmentation that maps source device spectrograms to resemble target-device distributions using unpaired training. A dedicated anechoic-chamber dataset with 18 devices and 75 sound types supports robust evaluation of cross-device performance and adaptation. The method achieves state-of-the-art generalization and adaptation gains, notably MC-200-Gen with an average F1 around 90.7% and low variance, and approaches Real-performance benchmarks in multi-device training settings. The study demonstrates practical implications for deploying SEC systems across diverse recording setups, while acknowledging limitations in one-to-one domain mappings and proposing future work on impulse-response integration and broader domain generalization.

Abstract

In this study, we introduce a new augmentation technique to enhance the resilience of sound event classification (SEC) systems against device variability through the use of CycleGAN. We also present a unique dataset to evaluate this method. As SEC systems become increasingly common, it is crucial that they work well with audio from diverse recording devices. Our method addresses limited device diversity in training data by enabling unpaired training to transform input spectrograms as if they are recorded on a different device. Our experiments show that our approach outperforms existing methods in generalization by 5.2% - 11.5% in weighted f1 score. Additionally, it surpasses the current methods in adaptability across diverse recording devices by achieving a 6.5% - 12.8% improvement in weighted f1 score.

Microphone Conversion: Mitigating Device Variability in Sound Event Classification

TL;DR

This work tackles the challenge of device variability in sound event classification (SEC) by introducing Microphone Conversion, a CycleGAN-based data augmentation that maps source device spectrograms to resemble target-device distributions using unpaired training. A dedicated anechoic-chamber dataset with 18 devices and 75 sound types supports robust evaluation of cross-device performance and adaptation. The method achieves state-of-the-art generalization and adaptation gains, notably MC-200-Gen with an average F1 around 90.7% and low variance, and approaches Real-performance benchmarks in multi-device training settings. The study demonstrates practical implications for deploying SEC systems across diverse recording setups, while acknowledging limitations in one-to-one domain mappings and proposing future work on impulse-response integration and broader domain generalization.

Abstract

In this study, we introduce a new augmentation technique to enhance the resilience of sound event classification (SEC) systems against device variability through the use of CycleGAN. We also present a unique dataset to evaluate this method. As SEC systems become increasingly common, it is crucial that they work well with audio from diverse recording devices. Our method addresses limited device diversity in training data by enabling unpaired training to transform input spectrograms as if they are recorded on a different device. Our experiments show that our approach outperforms existing methods in generalization by 5.2% - 11.5% in weighted f1 score. Additionally, it surpasses the current methods in adaptability across diverse recording devices by achieving a 6.5% - 12.8% improvement in weighted f1 score.
Paper Structure (15 sections, 2 equations, 6 figures, 2 tables)

This paper contains 15 sections, 2 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Dataset
  3. Sound Events
  4. Devices
  5. Environment
  6. Our Method
  7. CycleGAN
  8. Development Data
  9. Network Architecture
  10. Visualization of Generated Samples
  11. Experiments and Results
  12. Implementation Details
  13. Recent Approaches
  14. Results
  15. Conclusions and Limitations

Figures (6)

  • Figure 1: Spectrograms of the real and generated coughing sounds are presented in the top and bottom rows, respectively. The generated ones are produced by corresponding Microphone Conversion networks using a real spectrogram of Galaxy S22.
  • Figure 2: A difference spectra between a real spectrum (${x}_{source}$, iPhone 14) and a real/generated spectrum (${x}_{target}$, $\hat{x}_{target}$, Other devices). Spectra are calculated using Welch's method.
  • Figure 3: All sound
  • Figure 4: Whistle
  • Figure 5: All sound (To iPhone 14)
  • ...and 1 more figures