ODExAI: A Comprehensive Object Detection Explainable AI Evaluation

TL;DR

ODExAI addresses the lack of standardized evaluation for explainable AI in object detection by proposing a three dimensional framework—localization, faithfulness, and complexity. It operationalizes these dimensions with metrics such as Pointing Game (PG), Energy-Based Pointing Game (EBPG), Deletion/Insertion (Del/Ins), and Over-All (OA), plus Sparsity and Computation Time, and validates the framework on YOLOX and Faster R-CNN with MS COCO and PASCAL VOC. The demonstrative study shows clear trade offs between region based methods (strong faithfulness but high runtime) and CAM based methods (fast with excellent localization but lower faithfulness). ODExAI thus provides a principled, task specific approach to selecting XAI methods for object detection and offers public benchmarks and a roadmap for expanding evaluation tools.

Abstract

Explainable Artificial Intelligence (XAI) techniques for interpreting object detection models remain in an early stage, with no established standards for systematic evaluation. This absence of consensus hinders both the comparative analysis of methods and the informed selection of suitable approaches. To address this gap, we introduce the Object Detection Explainable AI Evaluation (ODExAI), a comprehensive framework designed to assess XAI methods in object detection based on three core dimensions: localization accuracy, faithfulness to model behavior, and computational complexity. We benchmark a set of XAI methods across two widely used object detectors (YOLOX and Faster R-CNN) and standard datasets (MS-COCO and PASCAL VOC). Empirical results demonstrate that region-based methods (e.g., D-CLOSE) achieve strong localization (PG = 88.49%) and high model faithfulness (OA = 0.863), though with substantial computational overhead (Time = 71.42s). On the other hand, CAM-based methods (e.g., G-CAME) achieve superior localization (PG = 96.13%) and significantly lower runtime (Time = 0.54s), but at the expense of reduced faithfulness (OA = 0.549). These findings demonstrate critical trade-offs among existing XAI approaches and reinforce the need for task-specific evaluation when deploying them in object detection pipelines. Our implementation and evaluation benchmarks are publicly available at: https://github.com/Analytics-Everywhere-Lab/odexai.

Figures (5)

• Figure 1: Overview of the architecture and web interface of our proposed Object Detection Explainable AI Evaluation (ODExAI).
• Figure 2: Demonstration of the localization property using region-specific relevance comparisons. Localization measures how well an explanation aligns with the spatial region that is most responsible for the model’s prediction. To evaluate this, a region of interest (ROI) is predefined in the input image (red box), marking the area where the most relevant evidence is expected to reside. In this example, the target is the object “train”. After model inference, an explanation map is generated, and pixel-wise relevance scores are computed. The $k$ relevance scores within the ROI are then compared with the top-$k$ scores from the full explanation map. A method with strong localization is expected to assign the highest relevance to pixels inside the ROI.
• Figure 3: Illustration of the faithfulness property using progressive perturbation. Faithfulness evaluates how the most relevant pixels affect the model’s prediction. Explanation scores are first ranked to identify the most relevant features, highlighted in red. Their corresponding positions are then mapped onto the flattened input, as indicated by the arrows. In the top panel, pixels are progressively removed (masked in black), while in the bottom panel, they are progressively added to a blank input. At each step, the modified input is passed through the model, and the prediction confidence is recorded. A faithful explanation produces a steep confidence drop when highly relevant pixels are removed and a sharp increase when they are added, whereas perturbing less relevant regions yields smaller changes.
• Figure 4: Visualization of the complexity property via relevance score distributions. Complexity measures how concentrated or dispersed an explanation is over the input space. After inference, relevance scores are assigned to all input pixels and sorted to generate a distribution curve. In the top panel, low complexity is illustrated by a steep curve where few pixels carry most of the relevance. In contrast, the bottom panel shows high complexity, with scores distributed more uniformly, resulting in a flatter curve. Low-complexity explanations are desirable because they highlight key regions, reduce the effective input space, and focus the analysis on essential model components, leading to more efficient and interpretable outputs.
• Figure 5: A visualization of the ODExAI framework for quantitative metrics of evaluated XAI methods for YOLOX on the MS COCO dataset: (a) representative metrics by categories (OA for Faithfulness, EBPG for Localization, and Sparsity for Complexity) and (b) all metrics.