Contextual Importance and Utility in Python: New Functionality and Insights with the py-ciu Package

TL;DR

The paper addresses explainability for tabular data by introducing py-ciu, a Python implementation of Contextual Importance and Utility (CIU) that explicitly separates Contextual Importance $CI_j$ and Contextual Utility $CU_j$ and computes contextual influence $\phi = CI \times (CU-\,CU_{ref})$. It emphasizes a model-agnostic, post-hoc approach with a sampling-based estimator for $y_{min}$ and $y_{max}$, enabling IO plots, Potential Influence plots, textual explanations, intermediate concepts, and contrastive explanations that offer capabilities beyond LIME/SHAP. Key contributions include a complete rewrite of CIU in Python to match R functionality, support for IC vocabularies, and a suite of CIU-specific visualizations and narrative explanations, demonstrated on Titanic/Johnny D and Ames/Boston datasets. The work advances practical XAI by delivering interpretable, contrastive, and globally oriented explanations, with a roadmap for extending CIU to natural language, time series, and interactive, co-constructive dialog with explainees through ICs.

Abstract

The availability of easy-to-use and reliable software implementations is important for allowing researchers in academia and industry to test, assess and take into use eXplainable AI (XAI) methods. This paper describes the \texttt{py-ciu} Python implementation of the Contextual Importance and Utility (CIU) model-agnostic, post-hoc explanation method and illustrates capabilities of CIU that go beyond the current state-of-the-art that could be useful for XAI practitioners in general.

Figures (11)

• Figure 1: Illustration of how CI and CU values are calculated for the input-output pair $x_{1}$ and $y_{1}$.
• Figure 2: Input-Output plots showing the probability of survival as a function of the numeric feature Age, the categorical feature Sibsp (number of siblings) and 3D plot of them jointly. CIU illustration has been included in the 2D plots by setting the parameter illustrate_CIU=True.
• Figure 3: Influence explanations with CIU (contextual influence), SHAP and LIME for Johnny D's probability of survival (61%).
• Figure 4: PI plot for 8-year old boy's probability of survival (61%).
• Figure 5: PI plot using bar length for visualizing CI and color for visualising CU, for 8-year old boy's probability of survival (61%).

Theorems & Definitions (3)

• Definition 1: Contextual Importance (CI)
• Definition 2: Contextual Utility (CU)
• Definition 3: Contextual influence