Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance

TL;DR

This work proposes a novel approach to explain the behavior of a black-box model under feature shifts by attributing an estimated performance change to interpretable input characteristics and refers to the method that combines concepts from Optimal Transport and Shapley Values as Explanatory Performance Estimation (XPE).

Abstract

Monitoring and maintaining machine learning models are among the most critical challenges in translating recent advances in the field into real-world applications. However, current monitoring methods lack the capability of provide actionable insights answering the question of why the performance of a particular model really degraded. In this work, we propose a novel approach to explain the behavior of a black-box model under feature shifts by attributing an estimated performance change to interpretable input characteristics. We refer to our method that combines concepts from Optimal Transport and Shapley Values as Explanatory Performance Estimation (XPE). We analyze the underlying assumptions and demonstrate the superiority of our approach over several baselines on different data sets across various data modalities such as images, audio, and tabular data. We also indicate how the generated results can lead to valuable insights, enabling explanatory model monitoring by revealing potential root causes for model deterioration and guiding toward actionable countermeasures.

Figures (7)

• Figure 1: Illustration of an opaque vision system subject to hardware degradation or environment changes, e.g., speckles on the lens or stray light. Explanatory Performance Estimation (XPE) allows to anticipate and explain the resulting performance decrease by highlighting which parts of the shift are harmful. This provides actionable insights to restore performance and facilitate effective model maintenance.
• Figure 2: Overview of our proposed framework for Explanatory Performance Estimation (XPE): a) Based on Optimal Transport, an optimal coupling is estimated to sample-wise align empirical source and target distributions. b) For a given target sample $x_t$ the conditional coupling $\hat{\pi}(X_s|x_t)$ indicates the most likely version of $x_t$ in the source domain denoted by $\hat{T}^{-1}(x_t)$. c) Given pre and post-shift version $\hat{T}^{-1}(x_t)$ and $x_t$ one can restrict shifts to individual input feature subsets $K_i$ and d) simulate partial feature shifts $\hat{T}_{K_i^c}^{-1}(x_t)$ by replacing $x_t$ with $\hat{T}^{-1}(x_t)$ outside the considered regions. e) Finally, all simulated partial feature shifts can be aggregated to quantify how individual feature shifts have contributed to the anticipated model loss based on Shapley Values.
• Figure 3: Feature shift importance on MNIST for local and global corruptions. The explanations given by XPE are most intuitive, only highlighting the area of the zigzag connecting the top parts of the '4', which changes the model's prediction to a '9'. A similar observation can be made for the spatter corruption, changing the prediction from '9' $\rightarrow$ '7'. XPE also allows uncovering the image regions shifting the prediction from '2' $\rightarrow$ '8' given a global increase in brightness.
• Figure 4: Feature shift importance for pneumonia detection dataset (PneumM) consisting of real chest X-rays. Neither image shows pneumonia. Only XPE consistently assigns the contribution to the loss increase caused by the shift to areas that might resemble pneumonia indicators (white spots) within the lung area.
• Figure 5: Comparison of XPE for zigzag ('6' $\rightarrow$ '0'), spatter ('6' $\rightarrow$ '5'), and brightness ('2' $\rightarrow$ '8') corruptions between a LeNet and an MLP model.

Theorems & Definitions (3)

• definition 1
• theorem 1
• lemma 1