Unifying VXAI: A Systematic Review and Framework for the Evaluation of Explainable AI

TL;DR

The paper addresses the lack of standardized evaluation for Explainable AI by introducing VXAI, a unified, three-dimension framework (desiderata, explanation type, contextuality) built on a PRISMA-guided systematic review. It aggregates $362$ metric proposals into $41$ aggregated metrics and provides a comprehensive taxonomy that spans multiple explanation types (FA, CE, ExE, WBS, NLE) and evaluation contexts. Key contributions include the detailed categorization scheme, an extensible metric catalog, and practical guidance to select complementary metrics, all supported by an interactive portal. The work stands to improve comparability and extensibility in VXAI evaluation, enabling more rigorous cross-study benchmarking and facilitating future standardization across domains and tasks.

Abstract

Modern AI systems frequently rely on opaque black-box models, most notably Deep Neural Networks, whose performance stems from complex architectures with millions of learned parameters. While powerful, their complexity poses a major challenge to trustworthiness, particularly due to a lack of transparency. Explainable AI (XAI) addresses this issue by providing human-understandable explanations of model behavior. However, to ensure their usefulness and trustworthiness, such explanations must be rigorously evaluated. Despite the growing number of XAI methods, the field lacks standardized evaluation protocols and consensus on appropriate metrics. To address this gap, we conduct a systematic literature review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and introduce a unified framework for the eValuation of XAI (VXAI). We identify 362 relevant publications and aggregate their contributions into 41 functionally similar metric groups. In addition, we propose a three-dimensional categorization scheme spanning explanation type, evaluation contextuality, and explanation quality desiderata. Our framework provides the most comprehensive and structured overview of VXAI to date. It supports systematic metric selection, promotes comparability across methods, and offers a flexible foundation for future extensions.

Figures (11)

• Figure 1: XAI evaluation classified into human-grounded and functionality-grounded evaluation, adapted from the classification framework by doshi2017towards and its visualization by zhou2021evaluating.
• Figure 2: The heatmaps show two alternative example user1explanans \ref{['\\glsentryuseriexplanans']}\ref{['fn:terminology']}, indicating which input regions were deemed decisive for the model's decision (the user1explanandum \ref{['\\glsentryuseriexplanandum']}\ref{['fn:terminology']}). A qualitative inspection allows for multiple interpretations, as it is unclear whether a) both the model and the explanation process ( user1explanation \ref{['\\glsentryuseriexplanation']}\ref{['fn:terminology']}) are correct or flawed (top), or b) one is correct and the other failed (bottom). The green checkmark marks the only scenario in which a human relying on visual plausibility would arrive at a correct conclusion; the red crosses indicate cases where such qualitative judgment would be misleading.
• Figure 3: Our search strategy building upon the user1prisma \ref{['\\glsentryuseriprisma']} guidelines page2021prisma. Notably, we split the process into an initial database search phase and a snowballing phase, identifying the most relevant literature in the second phase.
• Figure 4: Overview of the distribution of metrics within the categorization scheme. Each metric may be associated with multiple desiderata and user1explanation \ref{['\\glsentryuseriexplanation']} types, but only a single level of contextuality. Light-colored bars indicate partial alignment with a desideratum. For user1explanation \ref{['\\glsentryuseriexplanation']} types, light bars denote cases where no usage has been reported in the literature, though the metric is considered adaptable. The bottom histogram shows the number of metrics grouped by their reference count.
• Figure 5: Overview of the $41$ identified metrics, grouped by contextuality. Each metric is represented by a horizontal bar indicating the number of supporting references. Metrics are mapped to their associated desiderata, with arrow colors denoting the corresponding user1explanation \ref{['\\glsentryuseriexplanation']} type. These associations do not distinguish between full (✓) and partial (✓) alignment (unlike \ref{['tab:metric_overview']} ). For each desideratum, a pie chart summarizes the distribution of linked metrics by explanation type.