Making deep neural networks right for the right scientific reasons by interacting with their explanations

TL;DR

The novel learning setting of explanatory interactive learning is introduced and its benefits on a plant phenotyping research task are illustrated and it is demonstrated that explanatory interactiveLearning can help to avoid Clever Hans moments in machine learning.

Abstract

Deep neural networks have shown excellent performances in many real-world applications. Unfortunately, they may show "Clever Hans"-like behavior -- making use of confounding factors within datasets -- to achieve high performance. In this work, we introduce the novel learning setting of "explanatory interactive learning" (XIL) and illustrate its benefits on a plant phenotyping research task. XIL adds the scientist into the training loop such that she interactively revises the original model via providing feedback on its explanations. Our experimental results demonstrate that XIL can help avoiding Clever Hans moments in machine learning and encourages (or discourages, if appropriate) trust into the underlying model.

Figures (7)

• Figure 1: Explanatory Interactive Learning (XIL)---Human users revise learning machines towards trustworthy decision strategies. The cartoons sketch the main unerlying idea for each case (row). (a-left) Data samples, expert-classifications (checks and Xs with colors indicating the class) and explanations (overlaid with an edge filtered original image for better interpretability) that an expert expects of an ML model. Yellow corresponds to relevant regions, blue to irrelevant regions for a classification. Not even an expert can be certain about potential samples from a early disease stage and what a valid explanation should be. (a-middle) Illustration of hyperspectral data consisting of spatial and spectral dimensions. The planes on the top and left sides of the cube correspond to slices taken from the center of the cube but placed on the edges for visualization. (a-right) The characteristic reflectance of healthy tissue vs. disease spots. The vertical red, green and blue lines depict the three wavelengths of the RGB dataset. (b,c) Classifications of a deep neural network (b) and its explanations (c). The learned model clearly uses confounding factors, identified as the embedding agar solution, to explain its decision. (d) The human user provides feedback on the reasons. In turn, the machine gets new information and can continue learning. The human-revised deep network yields classifications matching a biologically plausible strategies. (All shown RGB images correspond to real RGB images, while the edge overlays resulted from pseudo-RGB images generated from the original hyperspectral dataset, cf. Methods RGB/HS classification.)
• Figure 2: XIL helps avoiding Clever Hans moments on unseen PASCAL VOC images (a). Ignoring user feedback, the model focuses on a source tag present in the lower left corner (b). Training it via interacting with its explanations, it does not consider the source tag to be relevant anymore (c). The visual explanations in (b,c) show relevant regions for the model's decision using light and irrelevant ones using dark colors.
• Figure 3: Spectral signatures of measured agar plates with sugar beet leaf discs. Signatures were extracted of agar on which healthy and inoculated sugar beet leaf discs were placed (a), of healthy and inoculated sugar beet leaf discs (b) and C. beticola symptoms of sugar beet leaves (c). Signatures were extracted from 100 pixels for each group and the mean value is presented. The vertical (green, blue, red) lines correspond to the wavelength selected for the pseudo-RGB images.
• Figure 4: Cluster analysis of the different decision strategies after training CNNs on the HS data with the cross-entropy loss (Default) in (a) and with the rrr loss in (b). The images are visualized in a two-dimensional t-SNE embedding and colored by the spectral clustering assignments.
• Figure 5: Results of the user study on trust development. (a) shows the total TiA Score over the three test conditions and (b-d) show in detail trust development (Q1) in correct rule learning after the three different learning stages of model accuracy (50%, 75%, 100%) for each test condition. Only statistically significant results are highlighted. The center line of the box plots represents the median of the data, the box the interquartile distance between the first and third quartile, the whiskers the minimum and maximum value, discarding outliers which are plotted individually above the whiskers. The number of asterisks indicate the P values: $*$$P \leq 0.05$, $**$$P\leq 0.01$, $***$$P \leq 0.001$.