ALOHa: A New Measure for Hallucination in Captioning Models

TL;DR

ALOHa introduces an open-vocabulary hallucination metric for captioning by leveraging an LLM to extract groundable objects, semantically matching candidate and reference objects with SBERT embeddings, and solving a maximum-similarity assignment via the Hungarian method. It yields per-object scores $ALOHa_o$ and a caption score $ALOHa$, enabling reliable and localizable detection that generalizes beyond fixed object sets. Evaluations on HAT and nocaps-FOIL show improvements over CHAIR and competitive performance against CLIP-based metrics, with notable gains in localization accuracy. While promising, the framework incurs higher compute and cost, hinges on reference captions, and faces nondeterminism and potential bias, suggesting directions for open-source LLM improvements and bias mitigation.

Abstract

Despite recent advances in multimodal pre-training for visual description, state-of-the-art models still produce captions containing errors, such as hallucinating objects not present in a scene. The existing prominent metric for object hallucination, CHAIR, is limited to a fixed set of MS COCO objects and synonyms. In this work, we propose a modernized open-vocabulary metric, ALOHa, which leverages large language models (LLMs) to measure object hallucinations. Specifically, we use an LLM to extract groundable objects from a candidate caption, measure their semantic similarity to reference objects from captions and object detections, and use Hungarian matching to produce a final hallucination score. We show that ALOHa correctly identifies 13.6% more hallucinated objects than CHAIR on HAT, a new gold-standard subset of MS COCO Captions annotated for hallucinations, and 30.8% more on nocaps, where objects extend beyond MS COCO categories. Our code is available at https://davidmchan.github.io/aloha/.

Figures (11)

• Figure 1: (Top) The SOTA prior object hallucination metric, CHAIR, is limited to MS COCO objects, and fails to detect the hallucinations in this image caption while ALOHa (ours, bottom) correctly assigns low similarity scores to the hallucinations "baseball player" and "bat". ALOHa does not penalize the caption for "catcher", "umpire", and "bass drum", as the caption indicates uncertainty of their presence.
• Figure 2: Overview of ALOHa. We prompt an LLM to extract visually grounded nouns from a candidate's machine-generated description and a set of references. We consider uncertain language (e.g., "goat or sheep"), add reference objects with and without modifiers (e.g., both "field" and "grassy field"), and avoid non-visual nouns (e.g., "picture" and "background"). Then, we compute a maximum-similarity linear assignment between candidate and reference object sets, the weights of which form the ALOHa$_\text{o}$. Matched pairs with low ALOHa$_\text{o}$ are likely hallucinations (e.g., "black cat", ALOHa$_\text{o}$$=0.2$). We additionally output the minimum ALOHa$_\text{o}$ as a caption-level ALOHa score.
• Figure 3: Qualitative Flickr30k examples. (Left) ALOHa correctly assigns low scores to the hallucinated "nun" and "bread", whereas CHAIR does not detect any hallucinations. (Right) Although ALOHa assigns high similarity between the hallucinated "electric guitar" and reference "(acoustic) guitar", it assigns low scores to the other 3 hallucinations. CHAIR detects only the hallucination "chair", missing the others.
• Figure 4: Performance on HAT validation set filtered for hallucinated objects, when comparing embedding methods and object extraction approaches.
• Figure A1: The prompt that we use for parsing objects from both captions and sets of reference captions.