NormEnsembleXAI: Unveiling the Strengths and Weaknesses of XAI Ensemble Techniques

TL;DR

This work addresses the need for robust, interpretable AI by evaluating XAI ensemble methods and introducing NormEnsembleXAI, a normalization-aware ensembling approach that combines explanations using min, max, or mean. It formalizes the normalization of explanations via Normal Standardization, Robust Standardization, and Second Moment Scaling, and demonstrates how these Normalized explanations can be fused into a single ensemble attribution. The authors compare NormEnsembleXAI to state-of-the-art methods like SupervisedXAI and Autoweighted across ImageNet-S and COCO, revealing that simple, well-normalized averaging can achieve competitive performance with lower resource demands. They also provide EnsembleXAI, an open-source PyTorch-compatible library to facilitate practical adoption, highlighting normalization as a key determinant of ensemble effectiveness and suggesting directions for future work, including CNN-based ensembling and domain generalization to text and tabular data.

Abstract

This paper presents a comprehensive comparative analysis of explainable artificial intelligence (XAI) ensembling methods. Our research brings three significant contributions. Firstly, we introduce a novel ensembling method, NormEnsembleXAI, that leverages minimum, maximum, and average functions in conjunction with normalization techniques to enhance interpretability. Secondly, we offer insights into the strengths and weaknesses of XAI ensemble methods. Lastly, we provide a library, facilitating the practical implementation of XAI ensembling, thus promoting the adoption of transparent and interpretable deep learning models.

Figures (4)

• Figure 1: Visualization of the proposed NormEnsembleXAI method. The NormEnsembleXAI method is designed to handle the diverse value ranges produced by various explanation methods. To address this, the algorithm employs normalization techniques, including Second Moment Scaling, Normal Standardization, or Robust Standardization. Subsequently, it utilizes aggregation functions for ensembling explanations, including Maximum, Minimum, and Mean.
• Figure 2: Visualization of the SupervisedXAI method Zou2022. Explanations for an instance are reshaped and concatenated into a matrix, which constitutes the training dataset X. The segmentation mask is similarly reshaped into a one-dimensional vector, serving as a set of labels Y. A multioutput Kernel Ridge Regression (KRR) model is then trained to predict the values of Y, using the explanations transformed into the X matrix as input.
• Figure 3: Illustration of the Autoweighted method Bobek2021. In this approach, explanations are assessed using a chosen metric, and subsequently, an Ensemble Score (ES) is computed for each explanation method. The final XAI ensemble is constructed as a weighted mean of normalized explanations, with individual weights determined by their respective ES values.
• Figure 4: Examples of XAI ensembling results. Each column displays the following images: the original image, XAI ensembling obtained with the supervised method, XAI ensembling obtained with the Autoweighted method, and our result using NormEnsembleXAI. In our approach, we employ average aggregation and utilize two distinct normalization techniques: Normal Standardization and Second Moment Scaling.