Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models

Minchan Kim, Minyeong Kim, Junik Bae, Suhwan Choi, Sungkyung Kim, Buru Chang

TL;DR

ESREAL is a novel unsupervised learning framework designed to suppress the generation of hallucinations through accurate localization and penalization of hallucinated tokens through accurate localization and penalization of hallucinated tokens.

Abstract

Hallucinations in vision-language models pose a significant challenge to their reliability, particularly in the generation of long captions. Current methods fall short of accurately identifying and mitigating these hallucinations. To address this issue, we introduce ESREAL, a novel unsupervised learning framework designed to suppress the generation of hallucinations through accurate localization and penalization of hallucinated tokens. Initially, ESREAL creates a reconstructed image based on the generated caption and aligns its corresponding regions with those of the original image. This semantic reconstruction aids in identifying both the presence and type of token-level hallucinations within the generated caption. Subsequently, ESREAL computes token-level hallucination scores by assessing the semantic similarity of aligned regions based on the type of hallucination. Finally, ESREAL employs a proximal policy optimization algorithm, where it selectively penalizes hallucinated tokens according to their token-level hallucination scores. Our framework notably reduces hallucinations in LLaVA, InstructBLIP, and mPLUG-Owl2 by 32.81%, 27.08%, and 7.46% on the CHAIR metric. This improvement is achieved solely through signals derived from the image itself, without the need for any image-text pairs.

ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models

TL;DR

ESREAL is a novel unsupervised learning framework designed to suppress the generation of hallucinations through accurate localization and penalization of hallucinated tokens through accurate localization and penalization of hallucinated tokens.

Abstract

Hallucinations in vision-language models pose a significant challenge to their reliability, particularly in the generation of long captions. Current methods fall short of accurately identifying and mitigating these hallucinations. To address this issue, we introduce ESREAL, a novel unsupervised learning framework designed to suppress the generation of hallucinations through accurate localization and penalization of hallucinated tokens. Initially, ESREAL creates a reconstructed image based on the generated caption and aligns its corresponding regions with those of the original image. This semantic reconstruction aids in identifying both the presence and type of token-level hallucinations within the generated caption. Subsequently, ESREAL computes token-level hallucination scores by assessing the semantic similarity of aligned regions based on the type of hallucination. Finally, ESREAL employs a proximal policy optimization algorithm, where it selectively penalizes hallucinated tokens according to their token-level hallucination scores. Our framework notably reduces hallucinations in LLaVA, InstructBLIP, and mPLUG-Owl2 by 32.81%, 27.08%, and 7.46% on the CHAIR metric. This improvement is achieved solely through signals derived from the image itself, without the need for any image-text pairs.
Paper Structure (49 sections, 7 equations, 15 figures, 9 tables)

This paper contains 49 sections, 7 equations, 15 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vision-Language Models
  4. Hallucination Mitigation in VLMs
  5. Hallucination Evaluation in VLMs
  6. Method
  7. Overview of ESREAL
  8. Reference-free Hallucination Detection Pipeline
  9. Semantic Reconstruction Module
  10. Alignment Module
  11. Scoring Module
  12. Non-existent Objects.
  13. Unfaithful Object Attributes.
  14. Inaccurate Relationships.
  15. Fine-grained PPO
  16. ...and 34 more sections

Figures (15)

  • Figure 1: Motivation of our study. Hallucinated tokens within the caption lead to semantic misalignment between the original and reconstructed images. By comparing the disparities among corresponding regions in the images, we can effectively identify and localize the hallucinated tokens within the caption.
  • Figure 2: Overview of ESREAL. The semantic reconstruction module, alignment module, and scoring module form a reference-free hallucination detection pipeline. Penalties produced by the detection pipeline for object, attribute, and relationship hallucinations are denoted as $p_\text{obj}$, $p_\text{attr}$, $p_\text{rel}$, respectively. Penalties are allocated to the corresponding hallucinated tokens, with a holistic reconstruction reward $r_\text{rec}$ allocated at the end of the caption.
  • Figure 3: Stability analysis. (a) a bar chart illustrating the win rates associated with different hallucination types. (b) a positive correlation between win rates and the number of images generated by the reconstruction module per caption.
  • Figure 4: A case study of reward allocation on hallucinated captions generated by LLaVA.
  • Figure 5: A case study comparing the captions generated before and after training LLaVA with ESREAL.
  • ...and 10 more figures