A Typology for Exploring the Mitigation of Shortcut Behavior

TL;DR

This work provides a unification of various XIL methods into a single typology by establishing a common set of basic modules, paving the way for a principled comparison of existing, but, importantly, also future XIL approaches.

Abstract

As machine learning models become increasingly larger, trained weakly supervised on large, possibly uncurated data sets, it becomes increasingly important to establish mechanisms for inspecting, interacting, and revising models to mitigate learning shortcuts and guarantee their learned knowledge is aligned with human knowledge. The recently proposed XIL framework was developed for this purpose, and several such methods have been introduced, each with individual motivations and methodological details. In this work, we provide a unification of various XIL methods into a single typology by establishing a common set of basic modules. In doing so, we pave the way for a principled comparison of existing, but, importantly, also future XIL approaches. In addition, we discuss existing and introduce novel measures and benchmarks for evaluating the overall abilities of a XIL method. Given this extensive toolbox, including our typology, measures, and benchmarks, we finally compare several recent XIL methods methodologically and quantitatively. In our evaluations, all methods prove to revise a model successfully. However, we found remarkable differences in individual benchmark tasks, revealing valuable application-relevant aspects for integrating these benchmarks in developing future methods.

Figures (7)

• Figure 1: The XIL Typology. Flowchart visualizing Algorithm \ref{['alg:typology']}. $\textsc{Select}$ describes how samples $X$ from $\mathcal{N}$ are selected in XIL. $\textsc{Explain}$ depicts how the model provides insights into its reasoning process to the teacher. With $\textsc{Obtain}$, the teacher, in turn, observes whether the learner's prediction is right or wrong, especially if it is based on the right or wrong reason, and returns corrective feedback, if necessary. The (explanatory) corrections obtained are redirected back into the model's learning process with the $\textsc{Revise}$ module to correct the model behavior according to the user.
• Figure 2: Qualitative inspection of explanations. The first column on each side shows the original image, the second column shows the Vanilla model (no XIL) attribution maps, and the remaining columns show the attribution maps of a model with each XIL method. Each row represents an explanation method to visualize the model prediction. The color bar indicates the activation of attribution maps (yellow (1) represents max activation and white (0) min activation). On the left (right) results for DecoyMNIST (DecoyFMNIST).
• Figure 3: Evaluation of (a) interaction efficiency and (b) performance in unconfounding a pre-trained model. Each task is evaluated on the DecoyMNIST (left) and DecoyFMNIST (right) dataset. (a) Test accuracy [%] with different numbers of used feedback interactions. The more interactions, the better the performance. However, a smaller number of interactions already suffices. (b) Test accuracy [%] over time after XIL is applied to an already fooled model. All methods, except CDEP and RBR, can recover the test performance and overcome the confounder. Each value represents the mean performance cross-validated on 5 runs; the given confidence intervals represent the standard deviation; (left in c) the arrow indicates the curve drops to random performance.
• Figure 4: (left) An ISIC19 image with confounder (red patch). (middle) an RRR-revised model and (right) a HINT-revised model generate explanations for the image. The explanations are visualized with GradCAM. RRR helps discover yet unknown confounders (dark corners), and HINT reveals the potential of the reward strategy.
• Figure 5: (a) attribution masks and (b) counter examples (with strategy randomize) for an DecoyMNIST example. (a) The original images, here the digits "3" and "5", have a confounder, here a gray decoy square in the corner. Below the original images, there are a correct feedback masks (counter examples) for penalizing the wrong reason and a correct feedback mask/hint for rewarding the right reason. The right side shows arbitrary and incomplete feedback masks (counter examples) for the feedback robustness experiment.