Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities

TL;DR

CAOS addresses object hallucination in LVLMs by introducing a context-aware evaluation framework that fuses ground-truth objects, training-set statistics, and semantic relationships from captions. It couples LLM-augmented object detection and an ensemble LVLM oracle to identify out-of-domain objects, and analyzes sequential generation and semantic drivers through multiple CAOS scores: $CAOS_T$, $CAOS_X$, $CAOS_K$, $CAOS_{T/X}$, $CAOS_{X/K}$, and $CAOS_{avg}$. Experiments across five LVLMs on MSCOCO reveal that top-$3$ frequent training objects disproportionately drive CAOS$_K$, while ground-truth- and context-related hallucinations vary by model, with InstructBLIP showing strong precision and lower CAOS$_K$. The framework offers an interpretable toolkit to guide robust LVLM development, though it remains limited to object hallucination, relies on detection/oracle components, and was validated on a modest subset of data. Future work could extend CAOS to spatial, relational, and numerical hallucinations for a more holistic evaluation of multimodal understanding.

Abstract

Despite their impressive performance on multi-modal tasks, large vision-language models (LVLMs) tend to suffer from hallucinations. An important type is object hallucination, where LVLMs generate objects that are inconsistent with the images shown to the model. Existing works typically attempt to quantify object hallucinations by detecting and measuring the fraction of hallucinated objects in generated captions. Additionally, more recent work also measures object hallucinations by directly querying the LVLM with binary questions about the presence of likely hallucinated objects based on object statistics like top-k frequent objects and top-k co-occurring objects. In this paper, we present Context-Aware Object Similarities (CAOS), a novel approach for evaluating object hallucination in LVLMs using object statistics as well as the generated captions. CAOS uniquely integrates object statistics with semantic relationships between objects in captions and ground-truth data. Moreover, existing approaches usually only detect and measure hallucinations belonging to a predetermined set of in-domain objects (typically the set of all ground-truth objects for the training dataset) and ignore generated objects that are not part of this set, leading to under-evaluation. To address this, we further employ language model--based object recognition to detect potentially out-of-domain hallucinated objects and use an ensemble of LVLMs for verifying the presence of such objects in the query image. CAOS also examines the sequential dynamics of object generation, shedding light on how the order of object appearance influences hallucinations, and employs word embedding models to analyze the semantic reasons behind hallucinations. CAOS aims to offer a nuanced understanding of the hallucination tendencies of LVLMs by providing a systematic framework to identify and interpret object hallucinations.

Figures (6)

• Figure 1: Overview of the CAOS framework: The CAOS framework evaluates LVLMs by generating captions for images with known ground-truth object annotations. The captions may include both in-domain and out-of-domain objects, which could be real or hallucinated. Specific color coding identifies object types: blue for real in-domain objects, red for hallucinated in-domain objects, purple for real out-of-domain objects, and orange for hallucinated out-of-domain objects. The key components of the framework are highlighted in yellow: constructing an ordered list of objects in the caption using an LLM-augmented identification module, querying an oracle (an ensemble of LVLMs) to confirm the presence or absence of out-of-domain objects (isolated on the basis of known in-domain objects in the training dataset) as ground-truth annotations are unavailable, and finally calculating CAOS scores (best viewed in color).
• Figure 2: CAOS scores are calculated for in-domain and out-of-domain hallucinated objects which are respectively identified from the ordered list of generated objects using ground-truth annotations and oracle decisions (about presence or absence in the image). CAOS$_T$, CAOS$_X$, and CAOS$_{K}$ are calculated as the maximum cosine similarity between the embeddings of the hallucinated object and those of ground-truth objects, preceding objects in the generated caption, and top-$k$ frequent objects in the training dataset, respectively. Inputs to the CAOS calculation module are highlighted in yellow for clarity (best viewed in color).
• Figure 3: Comparison between all evaluated LVLMs on Precision, Recall, the average number of objects per generated caption (normalized with division by 5), 1 - CHAIR$_S$ (greater is better), POPE-F1, and CAOS$_{T/X}$, CAOS$_{X/K}$, as well as CAOS$_{avg}$ scores (normalized using multiplication by 2) using both GloVe and MiniLM-L6 embeddings (best viewed in color).
• Figure 4: Trend of CAOS$_{K}$ scores with $k$ varying from 1 to 10 (best viewed in color).
• Figure 5: Comparison of CAOS$_T$, CAOS$_X$, and CAOS$_{K}$ scores for all hallucinated objects, only MSCOCO in-domain hallucinated objects, non-MSCOCO hallucinated objects, and all objects barring the top-3 most frequent MSCOCO in-domain objects (best viewed in color).