Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMs

TL;DR

Aurelia tackles audio-visual reasoning in LLMs by enabling test-time reasoning distillation via a multi-agent framework that injects structured reasoning into AVLLMs without fine-tuning. The approach uses a Reasoning Generator, Summarizer, Evaluator, and Feedback loop to create a refined reasoning sequence $r^*$, which conditions the AVLLM via $s^*=\mathcal{M}(v,a,q,r^*)$. To evaluate reasoning capabilities, the authors introduce AVReasonBench, a 4500-sample AV benchmark across six tasks including AV-GeoIQ that tests commonsense, geo-cultural knowledge, humor, and compositional understanding. Empirical results on 18 AVLLMs show substantial improvements with Aurelia, including up to 100% relative gains, demonstrating the effectiveness and practicality of training-free reasoning distillation for real-world AV applications. The work also discusses limitations and future directions, such as leveraging open-source LLMs and exploring integration of reasoning during training.

Abstract

Recent advancements in reasoning optimization have greatly enhanced the performance of large language models (LLMs). However, existing work fails to address the complexities of audio-visual scenarios, underscoring the need for further research. In this paper, we introduce AURELIA, a novel actor-critic based audio-visual (AV) reasoning framework that distills structured, step-by-step reasoning into AVLLMs at test time, improving their ability to process complex multi-modal inputs without additional training or fine-tuning. To further advance AVLLM reasoning skills, we present AVReasonBench, a challenging benchmark comprising 4500 audio-visual questions, each paired with detailed step-by-step reasoning. Our benchmark spans six distinct tasks, including AV-GeoIQ, which evaluates AV reasoning combined with geographical and cultural knowledge. Evaluating 18 AVLLMs on AVReasonBench reveals significant limitations in their multi-modal reasoning capabilities. Using AURELIA, we achieve up to a 100% relative improvement, demonstrating its effectiveness. This performance gain highlights the potential of reasoning-enhanced data generation for advancing AVLLMs in real-world applications. Our code and data will be publicly released at: https: //github.com/schowdhury671/aurelia.

Figures (12)

• Figure 1: Effect of injecting reasoning steps.Aurelia enhances the ZS capabilities of audio-visual models (e.g., VideoLLaMA2). The conventional pipeline struggles in audio-visual comprehension, leading to incorrect responses. In contrast, Aurelia systematically breaks down the problem into intermediate reasoning steps, guiding the model toward more accurate and interpretable answer.
• Figure 2: Overview of Aurelia: Our proposed Aurelia consists of a multi-agent interactive framework that functions in sync and generates reasoning steps that are then distilled inside the target model. The input set consisting of the audio, video, and question is first fed into the reasoning generator agent, which generates an initial set of reasoning steps that provide a structured pathway to reach the final answer. These reasoning steps are synthesized into a detailed caption by a Summarizer agent. The Evaluator agent then outputs a score that measures the relevance of the caption with the input audio and video. A feedback mechanism then provides supervision to the Reasoning generator based on the evaluation score, which adjusts its output to maximize the evaluation score. This actor-critique framework continues until the evaluation score exceeds a specific threshold or the number of iterations are exhausted.
• Figure 3: Qualitative Visualizations. Figure shows the qualitative visualizations of effect of Aurelia's reasoning distillation on the final answer across four tasks. Compared to vanilla zero-shot inference, Aurelia augments the target model with reasoning capabilities, leading to the improved answers.
• Figure 4: Examples of Failure Cases. (Left)Aurelia fails to comprehend audio, focus on single modality i.e. video, leading to incorrect reasoning chain. (Right)Aurelia fails to comprehend the dynamics of the video.
• Figure 5: Performance comparison across tasks. The distillation of reasoning information in the VITA model via Aurelia enhances its performance across all the tasks.