Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TLDiffGAN: A Latent Diffusion-GAN Framework with Temporal Information Fusion for Anomalous Sound Detection

Chengyuan Ma, Peng Jia, Hongyue Guo, Wenming Yang

TL;DR

Unsupervised anomalous sound detection faces challenges in fully modeling normal sound distributions, especially with single-modality inputs. This paper introduces TLDiffGAN, a dual-branch framework that fuses log-Mel spectrogram reconstruction via a latent diffusion–GAN backbone with features from pretrained audio encoders on raw waveforms, complemented by the TMixup augmentation to emphasize boundary regions. The detector combines reconstruction-based and embedding-based strategies, and the approach is validated on the DCASE 2020 Task 2 benchmark, achieving state-of-the-art AUC and pAUC values and improved time-frequency localization of anomalies. The work advances ASD by delivering robust multimodal representations and interpretable anomaly localization, with practical implications for industrial monitoring and fault diagnosis.

Abstract

Existing generative models for unsupervised anomalous sound detection are limited by their inability to fully capture the complex feature distribution of normal sounds, while the potential of powerful diffusion models in this domain remains largely unexplored. To address this challenge, we propose a novel framework, TLDiffGAN, which consists of two complementary branches. One branch incorporates a latent diffusion model into the GAN generator for adversarial training, thereby making the discriminator's task more challenging and improving the quality of generated samples. The other branch leverages pretrained audio model encoders to extract features directly from raw audio waveforms for auxiliary discrimination. This framework effectively captures feature representations of normal sounds from both raw audio and Mel spectrograms. Moreover, we introduce a TMixup spectrogram augmentation technique to enhance sensitivity to subtle and localized temporal patterns that are often overlooked. Extensive experiments on the DCASE 2020 Challenge Task 2 dataset demonstrate the superior detection performance of TLDiffGAN, as well as its strong capability in anomalous time-frequency localization.

TLDiffGAN: A Latent Diffusion-GAN Framework with Temporal Information Fusion for Anomalous Sound Detection

TL;DR

Unsupervised anomalous sound detection faces challenges in fully modeling normal sound distributions, especially with single-modality inputs. This paper introduces TLDiffGAN, a dual-branch framework that fuses log-Mel spectrogram reconstruction via a latent diffusion–GAN backbone with features from pretrained audio encoders on raw waveforms, complemented by the TMixup augmentation to emphasize boundary regions. The detector combines reconstruction-based and embedding-based strategies, and the approach is validated on the DCASE 2020 Task 2 benchmark, achieving state-of-the-art AUC and pAUC values and improved time-frequency localization of anomalies. The work advances ASD by delivering robust multimodal representations and interpretable anomaly localization, with practical implications for industrial monitoring and fault diagnosis.

Abstract

Existing generative models for unsupervised anomalous sound detection are limited by their inability to fully capture the complex feature distribution of normal sounds, while the potential of powerful diffusion models in this domain remains largely unexplored. To address this challenge, we propose a novel framework, TLDiffGAN, which consists of two complementary branches. One branch incorporates a latent diffusion model into the GAN generator for adversarial training, thereby making the discriminator's task more challenging and improving the quality of generated samples. The other branch leverages pretrained audio model encoders to extract features directly from raw audio waveforms for auxiliary discrimination. This framework effectively captures feature representations of normal sounds from both raw audio and Mel spectrograms. Moreover, we introduce a TMixup spectrogram augmentation technique to enhance sensitivity to subtle and localized temporal patterns that are often overlooked. Extensive experiments on the DCASE 2020 Challenge Task 2 dataset demonstrate the superior detection performance of TLDiffGAN, as well as its strong capability in anomalous time-frequency localization.
Paper Structure (13 sections, 6 equations, 3 figures, 3 tables)

This paper contains 13 sections, 6 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. LDGAN Backbone
  4. Log-mel Spectrogram Enhancement
  5. Pre-trained Audio Encoder
  6. Detector
  7. Experiment and Results
  8. Experimental Setup
  9. Performance Comparison
  10. Comparison between Pre-trained Audio Encoder
  11. Ablation Study
  12. Localization of Spectrogram Regions
  13. Conclusion

Figures (3)

  • Figure 1: The main pipeline of our TLDiffGAN.
  • Figure 2: LDGAN Framework.
  • Figure 3: Anomaly localization results for normal and anomalous samples of the ToyCar machine in the test set. The first row represents the reconstructed spectrogram from the model, the second row shows the average spectrogram of the training set, and the third row displays the difference between the two. Brighter regions indicate a higher degree of anomaly.