Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Not in Sync: Unveiling Temporal Bias in Audio Chat Models

Jiayu Yao, Shenghua Liu, Yiwei Wang, Rundong Cheng, Lingrui Mei, Baolong Bi, Zhen Xiong, Xueqi Cheng

TL;DR

This work reveals a fundamental limitation in Large Audio-Language Models: they semantically understand audio content but poorly ground it in time, exhibiting a systematic temporal bias in event localization. The authors formalize this with the Temporal Bias Index $\text{TBI}$ and compare it against MAE across multiple datasets and four state-of-the-art LALMs, showing that bias grows nonlinearly with context length and depends on event duration and position. Through interpretability analyses of cross-modal attention, they identify competing early-structural priors and late-semantic grounding signals as the mechanism underlying temporal misalignment, underscoring the need for temporally robust architectures and supervision. The study provides a rigorous evaluation framework and actionable directions for improving temporal fidelity in audio-language systems, with implications for lecture indexing, multimedia retrieval, and time-sensitive reasoning tasks.

Abstract

Large Audio Language Models (LALMs) are increasingly applied to audio understanding and multimodal reasoning, yet their ability to locate when events occur remains underexplored. We present the first systematic study of temporal bias in LALMs, revealing a key limitation in their timestamp prediction. For example, when asked "At which second does the lecturer introduce the key formula?", models often predict timestamps that are consistently earlier or later than the ground truth. Through controlled experiments on timestamped datasets, we find that temporal bias (i) is prevalent across datasets and models, (ii) increases with audio length - even accumulating to tens of seconds in extended recordings, and (iii) varies across event types and positions. We quantify this effect with the Temporal Bias Index (TBI), measuring systematic misalignment in predicted event timings, and complement it with a visualization framework. Our findings highlight a fundamental limitation in current LALMs and call for the development of temporally robust architectures.

Not in Sync: Unveiling Temporal Bias in Audio Chat Models

TL;DR

This work reveals a fundamental limitation in Large Audio-Language Models: they semantically understand audio content but poorly ground it in time, exhibiting a systematic temporal bias in event localization. The authors formalize this with the Temporal Bias Index and compare it against MAE across multiple datasets and four state-of-the-art LALMs, showing that bias grows nonlinearly with context length and depends on event duration and position. Through interpretability analyses of cross-modal attention, they identify competing early-structural priors and late-semantic grounding signals as the mechanism underlying temporal misalignment, underscoring the need for temporally robust architectures and supervision. The study provides a rigorous evaluation framework and actionable directions for improving temporal fidelity in audio-language systems, with implications for lecture indexing, multimedia retrieval, and time-sensitive reasoning tasks.

Abstract

Large Audio Language Models (LALMs) are increasingly applied to audio understanding and multimodal reasoning, yet their ability to locate when events occur remains underexplored. We present the first systematic study of temporal bias in LALMs, revealing a key limitation in their timestamp prediction. For example, when asked "At which second does the lecturer introduce the key formula?", models often predict timestamps that are consistently earlier or later than the ground truth. Through controlled experiments on timestamped datasets, we find that temporal bias (i) is prevalent across datasets and models, (ii) increases with audio length - even accumulating to tens of seconds in extended recordings, and (iii) varies across event types and positions. We quantify this effect with the Temporal Bias Index (TBI), measuring systematic misalignment in predicted event timings, and complement it with a visualization framework. Our findings highlight a fundamental limitation in current LALMs and call for the development of temporally robust architectures.
Paper Structure (25 sections, 2 equations, 3 figures, 5 tables)

This paper contains 25 sections, 2 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Temporal Bias Phenomenon
  3. Quantifying Temporal Misalignment
  4. Temporal Bias Index
  5. Mean Absolute Error(MAE)
  6. Empirical Evidence of a Systematic Bias
  7. Systematic Analysis of Temporal Bias
  8. Experimental Setup
  9. Datasets and Experimental Design
  10. Models.
  11. Implementation and Data Processing
  12. Results and Analysis
  13. Effect of Audio Length
  14. Effect of Event Duration
  15. Effect of Event Position in Audio
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Semantic Success, Temporal Failure. A case study of an LALM tasked with summarizing and timestamping events. (Left) The model produces a semantically correct summary with highly inaccurate timings. (Right) Internal attention correctly focuses on the relevant audio segments, yet large temporal errors persist, highlighting the paradox: the model knows what was said, but is lost on when.
  • Figure 2: Normalized mean absolute deviation by event position.
  • Figure 3: Evolution of attention distribution across decoder layers. Each subplot displays the audio waveform (black), attention weights as a heatmap overlay (red), and the attention curve (blue). The six panels show attention for early (0, 5), mid (10, 15), and late (25, 30) decoder layers. (Early Layers): Attention exhibits a strong structural bias, disproportionately focusing on the sequence start, a phenomenon we term 'attention front-loading'. (Late Layers): The focus gradually shifts from this structural prior to the actual semantic content, with attention peaks correctly aligning with the key speech events.