Why Are You Wrong? Counterfactual Explanations for Language Grounding with 3D Objects

TL;DR

This work tackles object referent identification for 3D objects grounded in natural language and addresses explainability by generating counterfactual utterances when a model misclassifies. The proposed method combines a sampling step that mutates content words with a genetic-algorithm-based optimization, using two objectives: flip the predicted class and maximize semantic similarity to the original utterance. It introduces four sampling strategies, including context-aware large language model (LLM) based approaches, and evaluates on ShapeTalk with three backbones, revealing insights about model bias and the importance of contextualized perturbations. The findings offer practical guidance for improving natural language prompts in robotics and language-assisted design, and point to data augmentation and alternative similarity metrics as promising future directions.

Abstract

Combining natural language and geometric shapes is an emerging research area with multiple applications in robotics and language-assisted design. A crucial task in this domain is object referent identification, which involves selecting a 3D object given a textual description of the target. Variability in language descriptions and spatial relationships of 3D objects makes this a complex task, increasing the need to better understand the behavior of neural network models in this domain. However, limited research has been conducted in this area. Specifically, when a model makes an incorrect prediction despite being provided with a seemingly correct object description, practitioners are left wondering: "Why is the model wrong?". In this work, we present a method answering this question by generating counterfactual examples. Our method takes a misclassified sample, which includes two objects and a text description, and generates an alternative yet similar formulation that would have resulted in a correct prediction by the model. We have evaluated our approach with data from the ShapeTalk dataset along with three distinct models. Our counterfactual examples maintain the structure of the original description, are semantically similar and meaningful. They reveal weaknesses in the description, model bias and enhance the understanding of the models behavior. Theses insights help practitioners to better interact with systems as well as engineers to improve models.

Figures (4)

• Figure 1: Left: Success rate as ratio of samples for which at least one valid counterfactual utterance is generated. Right: Average latent cosine similarity of the best valid counterfactual per sample and generation.
• Figure 2: Majority vote results obtained from five repetitive but independent evaluation runs using GPT-4o-mini openai2024gpt4o as a human surrogate chiang2023CanLargeLanguageliu2023GEvalNLGEvaluation to analyze the grammatical correctness and the semantic similarity of the generated counterfactual utterances.
• Figure 3: Commonly replaced and inserted words by the Qwen sampler to generate counterfactuals for the ViT model.
• Figure 4: Qualitative results of our method. Given the original utterance, the model misclassifies the left object as the target. With our counterfactual utterances, the model correctly identifies the target object on the right. Our counterfactuals maintain the structure of the original utterance, are semantically similar, meaningful, and provide insights why a prediction was wrong. All counterfactuals were generated using Qwen as sampler and ViT as model.