Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Closing the Modality Reasoning Gap for Speech Large Language Models

Chaoren Wang, Heng Lu, Xueyao Zhang, Shujie Liu, Yan Lu, Jinyu Li, Zhizheng Wu

TL;DR

TARS is introduced, a reinforcement-learning framework that aligns text-conditioned and speech-conditioned trajectories through an asymmetric reward design and significantly narrows the modality reasoning gap and achieves state-of-the-art performance among 7B-scale Speech LLMs.

Abstract

Although speech large language models have achieved notable progress, a substantial modality reasoning gap remains: their reasoning performance on speech inputs is markedly weaker than on text. This gap could be associated with representational drift across Transformer layers and behavior deviations in long-chain reasoning. To address this issue, we introduce TARS, a reinforcement-learning framework that aligns text-conditioned and speech-conditioned trajectories through an asymmetric reward design. The framework employs two dense and complementary signals: representation alignment, which measures layer-wise hidden-state similarity between speech- and text-conditioned trajectories, and behavior alignment, which evaluates semantic consistency between generated outputs and reference text completions. Experiments on challenging reasoning benchmarks, including MMSU and OBQA, show that our approach significantly narrows the modality reasoning gap and achieves state-of-the-art performance among 7B-scale Speech LLMs.

Closing the Modality Reasoning Gap for Speech Large Language Models

TL;DR

TARS is introduced, a reinforcement-learning framework that aligns text-conditioned and speech-conditioned trajectories through an asymmetric reward design and significantly narrows the modality reasoning gap and achieves state-of-the-art performance among 7B-scale Speech LLMs.

Abstract

Although speech large language models have achieved notable progress, a substantial modality reasoning gap remains: their reasoning performance on speech inputs is markedly weaker than on text. This gap could be associated with representational drift across Transformer layers and behavior deviations in long-chain reasoning. To address this issue, we introduce TARS, a reinforcement-learning framework that aligns text-conditioned and speech-conditioned trajectories through an asymmetric reward design. The framework employs two dense and complementary signals: representation alignment, which measures layer-wise hidden-state similarity between speech- and text-conditioned trajectories, and behavior alignment, which evaluates semantic consistency between generated outputs and reference text completions. Experiments on challenging reasoning benchmarks, including MMSU and OBQA, show that our approach significantly narrows the modality reasoning gap and achieves state-of-the-art performance among 7B-scale Speech LLMs.
Paper Structure (31 sections, 7 equations, 3 figures, 4 tables)

This paper contains 31 sections, 7 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Speech LLMs.
  4. Modality Alignment and Reasoning Gap.
  5. Reinforcement Learning for Reasoning.
  6. Method
  7. Problem Formulation
  8. Reward Modeling
  9. Base Reward.
  10. Representation Alignment Reward.
  11. Behavior Alignment Reward.
  12. Reinforcement Learning Framework
  13. Modality Specific Normalization.
  14. Experiments
  15. Experimental Setup
  16. ...and 16 more sections

Figures (3)

  • Figure 1: Overview of our framework. We introduce a reinforcement learning approach for trajectory alignment by optimizing an asymmetric reward function composed of representation alignment and behavior alignment.
  • Figure 2: Sensitivity Analysis of Representation Reward Layers. Average audio accuracy on MMSU and OBQA across different layer groups.
  • Figure 3: Layer-wise Representation Alignment Analysis. Shaded areas indicate 95% confidence intervals.