VISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysis

TL;DR

VISLIX addresses robust validation of vision models, especially object detectors, by automatically discovering data slices without reliance on image metadata or predefined concepts and by producing natural language explanations. It integrates context-aware embeddings, vision-language models, and large language models within an interactive visual analytics interface to enable experts to explore slices and test hypotheses. Expert studies and use cases show VISLIX can identify coherent failure slices, generate actionable explanations, and support model improvement through targeted fine-tuning, indicating practical benefits for safety-critical CV deployments. The framework is model-agnostic and adaptable to advancing foundation models, with potential to enhance MLOps for real-world vision systems.

Abstract

Real-world machine learning models require rigorous evaluation before deployment, especially in safety-critical domains like autonomous driving and surveillance. The evaluation of machine learning models often focuses on data slices, which are subsets of the data that share a set of characteristics. Data slice finding automatically identifies conditions or data subgroups where models underperform, aiding developers in mitigating performance issues. Despite its popularity and effectiveness, data slicing for vision model validation faces several challenges. First, data slicing often needs additional image metadata or visual concepts, and falls short in certain computer vision tasks, such as object detection. Second, understanding data slices is a labor-intensive and mentally demanding process that heavily relies on the expert's domain knowledge. Third, data slicing lacks a human-in-the-loop solution that allows experts to form hypothesis and test them interactively. To overcome these limitations and better support the machine learning operations lifecycle, we introduce VISLIX, a novel visual analytics framework that employs state-of-the-art foundation models to help domain experts analyze slices in computer vision models. Our approach does not require image metadata or visual concepts, automatically generates natural language insights, and allows users to test data slice hypothesis interactively. We evaluate VISLIX with an expert study and three use cases, that demonstrate the effectiveness of our tool in providing comprehensive insights for validating object detection models.

Figures (14)

• Figure 1: Illustrations of True Positive (TP), False Positive (FP), and False Negative (FN) for a car detector and IoU computation. Left: $\textrm{IoU} > 0.5$ (correct). Middle: $0 \leq \textrm{IoU} < 0.5$ (incorrect).
• Figure 2: VISLIX workflow. Inputs: True Positives (TPs), False Positives (FPs), and False Negatives (FNs) from an object (car) detector and validation images. Slice finding: Identifying data slice via image embeddings of FPs and FNs and estimating slice metrics via TPs. Slice explanation: Producing free-text explanations for each slice based on individual explanations of FPs and FNs, leveraging foundational models. Slice exploration: A visualization system that integrates all slices and explanations, enabling slice examination and validation.
• Figure 3: Context-aware embedding generation.
• Figure 4: Region annotations.
• Figure 5: Explanation generation for an FP in a car detector. A: Three regions derived from the FP: detection region (DR), context region (CR), and intersection region (IR). B: LLaVA answers predefined questions regarding different regions. C: GPT uses answers from B to chat with LLaVA about the CR. D: GPT explains the FP based on all the acquired information.