Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision

TL;DR

<3-5 sentence high-level summary> Volcano addresses multimodal hallucination by introducing a self-feedback guided revision loop that generates natural language feedback from the initial image-grounded response to revise and decide on final answers. It trains a single multimodal model to perform critique, revision, and decision, using text-based visual cues collected from LLaVA-derived data and a proprietary LLM for feedback. The approach achieves state-of-the-art results on multimodal hallucination benchmarks (MMHal-Bench, POPE, GAVIE) and strong performance on understanding benchmarks (MM-Vet, MMBench), with qualitative evidence that feedback grounds more richly on the image. The work releases code, data, and models, offering a practical pathway to mitigate multimodal hallucination through self-guided refinement without extra reward models.

Abstract

Large multimodal models suffer from multimodal hallucination, where they provide incorrect responses misaligned with the given visual information. Recent works have conjectured that one of the reasons behind multimodal hallucination is due to the vision encoder failing to ground on the image properly. To mitigate this issue, we propose a novel approach that leverages self-feedback as visual cues. Building on this approach, we introduce Volcano, a multimodal self-feedback guided revision model. Volcano generates natural language feedback to its initial response based on the provided visual information and utilizes this feedback to self-revise its initial response. Volcano effectively reduces multimodal hallucination and achieves state-of-the-art on MMHal-Bench, POPE, and GAVIE. It also improves on general multimodal abilities and outperforms previous models on MM-Vet and MMBench. Through qualitative analysis, we show that Volcano's feedback is properly grounded on the image than the initial response. This indicates that Volcano can provide itself with richer visual information through feedback generation, leading to self-correct hallucinations. We publicly release our model, data, and code at https://github.com/kaistAI/Volcano}{github.com/kaistAI/Volcano

Figures (7)

• Figure 1: Overview of Volcano. This example illustrates the process undertaken by Volcano for a question in the MMHal-Bench dataset. Before giving the response, Volcano goes through a critique-revise-decide process. It critiques its initial response with natural language feedback, revises the response based on the feedback, and decides whether to accept the revised answer.
• Figure 2: Overall process of Volcano. Volcano is a multimodal self-feedback guided revision model that takes an image and a question and then generates an improved response based on the self-feedback.
• Figure 3: Data collection.
• Figure 4: Coverage of image features attended during initial response and feedback generation on a single MMHal-Bench instance. The image attention heatmaps depict how the model's attention is distributed across image features, considering either all tokens or a subset of tokens in the output. In the text attention heatmaps above, the intensity of each token's background indicates the attention weight magnitude to image features, with darker highlights signifying higher weights. In the image attention heatmaps below, outliers at or above the 0.995th quantile are shown with the highest color intensity.
• Figure 5: Average amount of attention to image features during the initial response (left) and feedback (right) generation. Attention weights are averaged across instances in MMHal-Bench where Volcano's revision enhances the initial response.