Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Representation Loss Minimization with Randomized Selection Strategy for Efficient Environmental Fake Audio Detection

Orchid Chetia Phukan, Girish, Mohd Mujtaba Akhtar, Swarup Ranjan Behera, Nitin Choudhury, Arun Balaji Buduru, Rajesh Sharma, S. R Mahadeva Prasanna

TL;DR

This work tackles the challenge of high-dimensional representations in environmental audio deepfake detection by showing that randomly selecting 40–50% of foundation-model representations can preserve or even improve performance while drastically reducing downstream parameters and inference time. The approach outperforms traditional dimensionality reduction methods (PCA, SVD, KPCA, GRP) across both audio and multimodal foundation models, with LanguageBind often yielding the best detection metrics. The findings imply a practical, transferable strategy for efficient EADD deployment and invite further theoretical investigation into representation redundancy. Overall, the paper demonstrates a simple yet powerful method to achieve near-SOTA accuracy with substantially lower computational cost.

Abstract

The adaptation of foundation models has significantly advanced environmental audio deepfake detection (EADD), a rapidly growing area of research. These models are typically fine-tuned or utilized in their frozen states for downstream tasks. However, the dimensionality of their representations can substantially lead to a high parameter count of downstream models, leading to higher computational demands. So, a general way is to compress these representations by leveraging state-of-the-art (SOTA) unsupervised dimensionality reduction techniques (PCA, SVD, KPCA, GRP) for efficient EADD. However, with the application of such techniques, we observe a drop in performance. So in this paper, we show that representation vectors contain redundant information, and randomly selecting 40-50% of representation values and building downstream models on it preserves or sometimes even improves performance. We show that such random selection preserves more performance than the SOTA dimensionality reduction techniques while reducing model parameters and inference time by almost over half.

Representation Loss Minimization with Randomized Selection Strategy for Efficient Environmental Fake Audio Detection

TL;DR

This work tackles the challenge of high-dimensional representations in environmental audio deepfake detection by showing that randomly selecting 40–50% of foundation-model representations can preserve or even improve performance while drastically reducing downstream parameters and inference time. The approach outperforms traditional dimensionality reduction methods (PCA, SVD, KPCA, GRP) across both audio and multimodal foundation models, with LanguageBind often yielding the best detection metrics. The findings imply a practical, transferable strategy for efficient EADD deployment and invite further theoretical investigation into representation redundancy. Overall, the paper demonstrates a simple yet powerful method to achieve near-SOTA accuracy with substantially lower computational cost.

Abstract

The adaptation of foundation models has significantly advanced environmental audio deepfake detection (EADD), a rapidly growing area of research. These models are typically fine-tuned or utilized in their frozen states for downstream tasks. However, the dimensionality of their representations can substantially lead to a high parameter count of downstream models, leading to higher computational demands. So, a general way is to compress these representations by leveraging state-of-the-art (SOTA) unsupervised dimensionality reduction techniques (PCA, SVD, KPCA, GRP) for efficient EADD. However, with the application of such techniques, we observe a drop in performance. So in this paper, we show that representation vectors contain redundant information, and randomly selecting 40-50% of representation values and building downstream models on it preserves or sometimes even improves performance. We show that such random selection preserves more performance than the SOTA dimensionality reduction techniques while reducing model parameters and inference time by almost over half.
Paper Structure (14 sections, 17 figures, 1 table)

This paper contains 14 sections, 17 figures, 1 table.

Table of Contents

  1. Introduction
  2. Foundation Models
  3. Audio Foundation Models
  4. Multimodal Foundation Models
  5. Dimensionality Reduction Optimization
  6. Baseline Dimensionality Reduction Techniques
  7. Random Selection Strategy
  8. Experiments
  9. Benchmark Dataset
  10. Training Details
  11. Experimental Results and Discussion
  12. Evaluation of Foundation Models
  13. Dimensionality Reduction Optimization
  14. Conclusion

Figures (17)

  • Figure 1: Modeling with a randomized selection strategy.
  • Figure 2: Barplot illustrating the impact of subset percentage (x-axis) on model parameters in millions (y-axis): (A) LFCC (13-dimension) , (B) MFCC (39-dimension); (C) LanguageBind, Unispeech-SAT, wavLM, wav2vec2 (768-dimension); (D) ImageBind, CLAP, TRILLsson (1024-dimension).
  • Figure 3: Inference Time Plot: x-axis represents percentage random selection of representation values; y-axis represents the inference time in seconds; LB, IB, UNI stands for LanguageBind, ImageBind, Unispeech-SAT respectively.
  • Figure : ImageBind
  • Figure : (a)
  • ...and 12 more figures