Evaluating Hallucinations in Multimodal LLMs with Spoken Queries under Diverse Acoustic Conditions

TL;DR

This study investigates how spoken queries affect hallucinations in multimodal LLMs under diverse acoustic conditions. It introduces RePOPE-Spk, an audio-augmented extension of the RePOPE benchmark, to evaluate robustness when queries are spoken rather than written. Across Gemini and Gemma, results show that spoken inputs increase hallucinations, with error rates rising by $3\%$ under clean speech and up to $20\%$ with environmental noise; input order and query length further modulate robustness, while many-shot prompting and chain-of-thought reasoning offer only partial mitigation. The findings reveal a critical gap in voice-enabled multimodal systems and advocate for developing methods to ensure reliable, speech-driven interaction in real-world deployments.

Abstract

Hallucinations in vision-language models have been extensively studied using benchmarks that probe reliability in image-text settings. In contrast, the effect of spoken queries on multimodal hallucinations remains largely unexplored, despite the growing role of voice-driven interfaces. In this work, we investigate how spoken input influences hallucinations in multimodal large language models. We present RePOPE-Spk, an audio-augmented extension of the RePOPE benchmark, where queries are provided as speech under diverse acoustic conditions. Using RePOPE-Spk, we systematically evaluate both proprietary and open-source models. Experimental results show that hallucinations escalate when queries are spoken rather than written: error rates increase by 3% under clean speech and by up to 20% with environmental noise. Input order and query length further affect robustness, while strategies such as many-shot prompting and chain-of-thought reasoning offer partial but insufficient mitigation. These findings highlight a critical and underexplored challenge, opening new directions for building reliable voice interface systems.

Figures (1)

• Figure 1: Illustration of how spoken queries amplify multimodal hallucinations: (a) image–text queries yield correct answers, (b) replacing text with speech with the same content increases hallucinations, and (c) adding noise to the speech query further exacerbates hallucinations.