Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Domain Audio Question Answering Toward Acoustic Content Reasoning in The DCASE 2025 Challenge

Chao-Han Huck Yang, Sreyan Ghosh, Qing Wang, Jaeyeon Kim, Hengyi Hong, Sonal Kumar, Guirui Zhong, Zhifeng Kong, S Sakshi, Vaibhavi Lokegaonkar, Oriol Nieto, Ramani Duraiswami, Dinesh Manocha, Gunhee Kim, Jun Du, Rafael Valle, Bryan Catanzaro

TL;DR

This paper introduces the DCASE 2025 Audio Question Answering (AQA) benchmark, a multi-domain task that benchmarks interactive reasoning over acoustic content across Bioacoustics, Temporal Soundscapes, and Complex QA. It defines dataset composition, evaluation protocols, and baseline zero-shot models to assess capabilities in perceiving, grounding, and reasoning about audio with external knowledge. The study reveals substantial cross-domain variation and highlights the challenge of achieving human-like audio understanding, motivating future work in targeted prompting, efficient fine-tuning, and robust multimodal inference. The benchmark and accompanying resources aim to accelerate progress toward general, context-aware audio-language agents capable of interpreting real-world acoustic scenes.

Abstract

We present Task 5 of the DCASE 2025 Challenge: an Audio Question Answering (AQA) benchmark spanning multiple domains of sound understanding. This task defines three QA subsets (Bioacoustics, Temporal Soundscapes, and Complex QA) to test audio-language models on interactive question-answering over diverse acoustic scenes. We describe the dataset composition (from marine mammal calls to soundscapes and complex real-world clips), the evaluation protocol (top-1 accuracy with answer-shuffling robustness), and baseline systems (Qwen2-Audio-7B, AudioFlamingo 2, Gemini-2-Flash). Preliminary results on the development set are compared, showing strong variation across models and subsets. This challenge aims to advance the audio understanding and reasoning capabilities of audio-language models toward human-level acuity, which are crucial for enabling AI agents to perceive and interact about the world effectively.

Multi-Domain Audio Question Answering Toward Acoustic Content Reasoning in The DCASE 2025 Challenge

TL;DR

This paper introduces the DCASE 2025 Audio Question Answering (AQA) benchmark, a multi-domain task that benchmarks interactive reasoning over acoustic content across Bioacoustics, Temporal Soundscapes, and Complex QA. It defines dataset composition, evaluation protocols, and baseline zero-shot models to assess capabilities in perceiving, grounding, and reasoning about audio with external knowledge. The study reveals substantial cross-domain variation and highlights the challenge of achieving human-like audio understanding, motivating future work in targeted prompting, efficient fine-tuning, and robust multimodal inference. The benchmark and accompanying resources aim to accelerate progress toward general, context-aware audio-language agents capable of interpreting real-world acoustic scenes.

Abstract

We present Task 5 of the DCASE 2025 Challenge: an Audio Question Answering (AQA) benchmark spanning multiple domains of sound understanding. This task defines three QA subsets (Bioacoustics, Temporal Soundscapes, and Complex QA) to test audio-language models on interactive question-answering over diverse acoustic scenes. We describe the dataset composition (from marine mammal calls to soundscapes and complex real-world clips), the evaluation protocol (top-1 accuracy with answer-shuffling robustness), and baseline systems (Qwen2-Audio-7B, AudioFlamingo 2, Gemini-2-Flash). Preliminary results on the development set are compared, showing strong variation across models and subsets. This challenge aims to advance the audio understanding and reasoning capabilities of audio-language models toward human-level acuity, which are crucial for enabling AI agents to perceive and interact about the world effectively.
Paper Structure (14 sections, 2 figures, 4 tables)

This paper contains 14 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Dataset Details
  3. Part 1: Bioacoustics QA (BQA)
  4. Part 2: Temporal Soundscapes QA (TSQA)
  5. Part 3: Complex QA (CQA)
  6. Dataset and Codebase Availability
  7. Evaluation
  8. Evaluation Metric
  9. Submission Guidelines
  10. Baseline Systems and Results
  11. Baseline Systems
  12. Evaluation Results
  13. Qualitative Analysis
  14. Conclusion

Figures (2)

  • Figure 1: The proposed audio question answering (AQA) dataset with three categories: Bioacoustics QA (BQA), Temporal Soundscapes QA (TSQA), and Complex QA (CQA).
  • Figure 2: Causal graphs illustrating key differences among three audio understanding tasks. Shaded (gray) nodes represent observable variables, while unshaded nodes represent latent or implicit factors. (a) In Audio Classification, models map audio recordings to predefined labels based solely on surface acoustic features. (b) Automated Audio Captioning introduces an intermediate representation involving interactive events and conceptual understanding to generate natural language descriptions. (c) In Audio Question Answering (AQA), reasoning depends not only on the audio and the question but also on latent confounding factors such as audio information, interactive events, and external knowledge. These factors enrich the reasoning process required to derive accurate answers.