Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Discovery of Interpretable Audio Attributes with Multimodal LLMs for Low-Resource Classification

Kosuke Yoshimura, Hisashi Kashima

TL;DR

This work proposes a method for adaptively discovering interpretable audio attributes using Multimodal Large Language Models (MLLMs), and achieves significantly faster attribute discovery by replacing humans in the AdaFlock framework with MLLMs.

Abstract

In predictive modeling for low-resource audio classification, extracting high-accuracy and interpretable attributes is critical. Particularly in high-reliability applications, interpretable audio attributes are indispensable. While human-driven attribute discovery is effective, its low throughput becomes a bottleneck. We propose a method for adaptively discovering interpretable audio attributes using Multimodal Large Language Models (MLLMs). By replacing humans in the AdaFlock framework with MLLMs, our method achieves significantly faster attribute discovery. Our method dynamically identifies salient acoustic characteristics via prompting and constructs an attribute-based ensemble classifier. Experimental results across various audio tasks demonstrate that our method outperforms direct MLLM prediction in the majority of evaluated cases. The entire training completes within 11 minutes, proving it a practical, adaptive solution that surpasses conventional human-reliant approaches.

Adaptive Discovery of Interpretable Audio Attributes with Multimodal LLMs for Low-Resource Classification

TL;DR

This work proposes a method for adaptively discovering interpretable audio attributes using Multimodal Large Language Models (MLLMs), and achieves significantly faster attribute discovery by replacing humans in the AdaFlock framework with MLLMs.

Abstract

In predictive modeling for low-resource audio classification, extracting high-accuracy and interpretable attributes is critical. Particularly in high-reliability applications, interpretable audio attributes are indispensable. While human-driven attribute discovery is effective, its low throughput becomes a bottleneck. We propose a method for adaptively discovering interpretable audio attributes using Multimodal Large Language Models (MLLMs). By replacing humans in the AdaFlock framework with MLLMs, our method achieves significantly faster attribute discovery. Our method dynamically identifies salient acoustic characteristics via prompting and constructs an attribute-based ensemble classifier. Experimental results across various audio tasks demonstrate that our method outperforms direct MLLM prediction in the majority of evaluated cases. The entire training completes within 11 minutes, proving it a practical, adaptive solution that surpasses conventional human-reliant approaches.
Paper Structure (17 sections, 1 figure, 4 tables, 1 algorithm)

This paper contains 17 sections, 1 figure, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setup
  4. Proposed Method
  5. Sampling and Attribute Definition via M_def
  6. Attribute Labeling via M_lab
  7. Training Weak Classifiers via Boosting
  8. Inference
  9. Experiments
  10. Datasets
  11. Experimental Setup
  12. Performance Comparison
  13. Qualitative Analysis: Discovered Attributes
  14. Analysis of MLLM Variations in Attribute Definition
  15. Training Time Analysis
  16. ...and 2 more sections

Figures (1)

  • Figure 1: An iterative framework for MLLM-based attribute discovery and weak classifier training. Sampling weights are adaptively updated based on prediction results to discover attributes and refine model performance.