CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillation

TL;DR

CORD tackles the persistent audio–text reasoning gap in Large Audio Language Models by introducing an in-model, on-policy cross-modal self-distillation framework. It combines token-level importance-weighted reverse KL alignment with a judge-guided sequence-level reward via Group Relative Policy Optimization to align audio-conditioned reasoning with text-conditioned behavior within a single model. The approach achieves substantial reductions in the modality gap using only about 80k synthetic samples, while preserving auxiliary audio capabilities and demonstrating cross-domain generalization. Ablation analyses confirm the necessity of both the on-policy, trajectory-level supervision and the fine-grained token weighting for stable, effective cross-modal alignment. Overall, CORD offers a scalable, data-efficient pathway to improve reasoning in multimodal LALMs and motivates further exploration with broader data and tasks.

Abstract

Large Audio Language Models (LALMs) have garnered significant research interest. Despite being built upon text-based large language models (LLMs), LALMs frequently exhibit a degradation in knowledge and reasoning capabilities. We hypothesize that this limitation stems from the failure of current training paradigms to effectively bridge the acoustic-semantic gap within the feature representation space. To address this challenge, we propose CORD, a unified alignment framework that performs online cross-modal self-distillation. Specifically, it aligns audio-conditioned reasoning with its text-conditioned counterpart within a unified model. Leveraging the text modality as an internal teacher, CORD performs multi-granularity alignment throughout the audio rollout process. At the token level, it employs on-policy reverse KL divergence with importance-aware weighting to prioritize early and semantically critical tokens. At the sequence level, CORD introduces a judge-based global reward to optimize complete reasoning trajectories via Group Relative Policy Optimization (GRPO). Empirical results across multiple benchmarks demonstrate that CORD consistently enhances audio-conditioned reasoning and substantially bridges the audio-text performance gap with only 80k synthetic training samples, validating the efficacy and data efficiency of our on-policy, multi-level cross-modal alignment approach.

Figures (5)

• Figure 1: The overall framework of CORD. Given semantically equivalent audio and text inputs, CORD performs on-policy cross-modal self-distillation within a single model. Audio-conditioned trajectories are aligned to text-conditioned behaviors at two levels: (i) a token-level objective that applies importance-aware and position-aware reverse KL weighting along audio rollouts, and (ii) a sequence-level objective that uses a judge-based reward optimized via GRPO to enforce global reasoning consistency.
• Figure 2: Token-level reverse KL visualization along audio-conditioned rollouts generated by Qwen2-Audio-7B-Instruct. Each block represents a generated token, annotated with its reverse KL divergence between text- and audio-conditioned distributions. The correct case shows low divergence on the final option token, while the incorrect case exhibits large reverse KL at the decision token and several early reasoning steps. Notably, the incorrect output contains formulaic phrases such as "We refer to $\cdots$," reflecting a hallucination-prone pattern under audio inputs.
• Figure 3: Tokens from high-KL regions.
• Figure 4: Tokens from low-KL (bottom) regions.
• Figure 6: Sensitivity analysis of the weighting intensity $\alpha$ and $\beta$. To reduce hyperparameter complexity, we set $\alpha = \beta$. The scores represent relative improvements over the baseline ($\alpha=\beta=1.0$). A value of $2.0$ yields the most consistent gains across all tasks.