Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Retrieval-Augmented Approach for Unsupervised Anomalous Sound Detection and Captioning without Model Training

Ryoya Ogura, Tomoya Nishida, Yohei Kawaguchi

TL;DR

The proposed method employs a retrieval-augmented approach for captioning of anomalous sounds and difference captioning in the embedding space output by the pre-trained CLAP (contrastive language-audio pre-training) model makes the anomalous sound detection results consistent with the captions and does not require training.

Abstract

This paper proposes a method for unsupervised anomalous sound detection (UASD) and captioning the reason for detection. While there is a method that captions the difference between given normal and anomalous sound pairs, it is assumed to be trained and used separately from the UASD model. Therefore, the obtained caption can be irrelevant to the differences that the UASD model captured. In addition, it requires many caption labels representing differences between anomalous and normal sounds for model training. The proposed method employs a retrieval-augmented approach for captioning of anomalous sounds. Difference captioning in the embedding space output by the pre-trained CLAP (contrastive language-audio pre-training) model makes the anomalous sound detection results consistent with the captions and does not require training. Experiments based on subjective evaluation and a sample-wise analysis of the output captions demonstrate the effectiveness of the proposed method.

Retrieval-Augmented Approach for Unsupervised Anomalous Sound Detection and Captioning without Model Training

TL;DR

The proposed method employs a retrieval-augmented approach for captioning of anomalous sounds and difference captioning in the embedding space output by the pre-trained CLAP (contrastive language-audio pre-training) model makes the anomalous sound detection results consistent with the captions and does not require training.

Abstract

This paper proposes a method for unsupervised anomalous sound detection (UASD) and captioning the reason for detection. While there is a method that captions the difference between given normal and anomalous sound pairs, it is assumed to be trained and used separately from the UASD model. Therefore, the obtained caption can be irrelevant to the differences that the UASD model captured. In addition, it requires many caption labels representing differences between anomalous and normal sounds for model training. The proposed method employs a retrieval-augmented approach for captioning of anomalous sounds. Difference captioning in the embedding space output by the pre-trained CLAP (contrastive language-audio pre-training) model makes the anomalous sound detection results consistent with the captions and does not require training. Experiments based on subjective evaluation and a sample-wise analysis of the output captions demonstrate the effectiveness of the proposed method.

Paper Structure

This paper contains 13 sections, 3 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. CLAP
  4. Retrieval-Augmented Audio Caption (RECAP)
  5. Proposed Method
  6. UASD with CLAP
  7. Difference caption generation for anomalous sounds
  8. Difference caption generation with predefined texts
  9. Experiment
  10. Experimental conditions
  11. UASD experimental results
  12. Captioning experimental results
  13. Conclusion

Figures (3)

  • Figure 1: UASD and captioning flow in our proposed method.
  • Figure 2: Violin plot for MOS values of each ID in each machine. "Text decoder", "Zero-shot", and "Hybrid" denote Text decoder-based method, Zero-shot classification-based method, and combination of these methods, respectively. White dot denotes the median and black bar denotes range of quartiles.
  • Figure 3: Spectrograms of anomalous and reference normal sounds and corresponding captions. Black box (first row) denotes individual captions for each sample; Red box (second row) denotes captions for CLAP's text decoder-based method; Blue box (third row) denotes captions for zero-shot classification-based method; Purple box (forth row) denotes captions for combination of both methods.