Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EvA: An Evidence-First Audio Understanding Paradigm for LALMs

Xinyuan Xie, Shunian Chen, Zhiheng Liu, Yuhao Zhang, Zhiqiang Lv, Liyin Liang, Benyou Wang

Abstract

Large Audio Language Models (LALMs) still struggle in complex acoustic scenes because they often fail to preserve task-relevant acoustic evidence before reasoning begins. We call this failure the evidence bottleneck: state-of-the-art systems show larger deficits in evidence extraction than in downstream reasoning, suggesting that the main limitation lies in upstream perception rather than reasoning policy. To address this problem, we propose EvA (Evidence-First Audio), a dual-path architecture that combines Whisper and CED-Base through non-compressive, time-aligned fusion. EvA first aggregates intermediate CED layers to preserve multi-scale acoustic cues, then aligns the aggregated CED features to the Whisper timeline and adds the two streams without changing sequence length. We also build EvA-Perception, a large-scale open-source training set with about 54K event-ordered captions (150 h) and about 500K QA pairs. Under a unified zero-shot protocol, EvA achieves the best open-source Perception scores on MMAU, MMAR, and MMSU, and improves over Kimi-Audio-7B on all reported metrics, with the largest gains on perception-heavy splits. These results support the evidence-first hypothesis: stronger audio understanding depends on preserving acoustic evidence before reasoning.

EvA: An Evidence-First Audio Understanding Paradigm for LALMs

Abstract

Large Audio Language Models (LALMs) still struggle in complex acoustic scenes because they often fail to preserve task-relevant acoustic evidence before reasoning begins. We call this failure the evidence bottleneck: state-of-the-art systems show larger deficits in evidence extraction than in downstream reasoning, suggesting that the main limitation lies in upstream perception rather than reasoning policy. To address this problem, we propose EvA (Evidence-First Audio), a dual-path architecture that combines Whisper and CED-Base through non-compressive, time-aligned fusion. EvA first aggregates intermediate CED layers to preserve multi-scale acoustic cues, then aligns the aggregated CED features to the Whisper timeline and adds the two streams without changing sequence length. We also build EvA-Perception, a large-scale open-source training set with about 54K event-ordered captions (150 h) and about 500K QA pairs. Under a unified zero-shot protocol, EvA achieves the best open-source Perception scores on MMAU, MMAR, and MMSU, and improves over Kimi-Audio-7B on all reported metrics, with the largest gains on perception-heavy splits. These results support the evidence-first hypothesis: stronger audio understanding depends on preserving acoustic evidence before reasoning.

Paper Structure

This paper contains 58 sections, 10 equations, 8 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Large Audio Language Models
  4. Audio Understanding Benchmarks
  5. Motivation: the Evidence Bottleneck
  6. Notation.
  7. Single-Path LALMs: An Information Constraint
  8. Dual-Path Architectures: More Complementary Evidence
  9. Implications for LALM Design
  10. Method: Evidence-First Audio Understanding Paradigm
  11. Architecture
  12. Complementary Dual Encoders.
  13. Hierarchical Evidence Aggregation.
  14. Time-Aware Alignment and Inject-and-Add Fusion.
  15. EvA-Perception: A Dataset for Evidence-Grounded Training
  16. ...and 43 more sections

Figures (8)

  • Figure 1: Perception--reasoning performance gap comparison between models and humans. Model performance is averaged over Qwen2.5– Omni and Kimi– Audio– 7B.
  • Figure 2: Model architecture. The left half of the left panel shows the Kimi-Audio backbone, while the right half illustrates the additional EvA Path modules. Audio is encoded into four frequency-band features by the CED Encoder, pooled across frequencies, fused via cross-attention, temporally aligned with Whisper, and integrated through gated additive fusion.
  • Figure 3: Qualitative comparison of captions generated by different models on AudioCaps examples.
  • Figure 4: Instruction for Caption Generation.
  • Figure 5: Instruction for QA Generation.
  • ...and 3 more figures