Hidden Monotonicity: Explaining Deep Neural Networks via their DC Decomposition

TL;DR

It is demonstrated that monotonicity can still be used in two ways to boost explainability, and it is exhibited that training a model as the difference between two monotone neural networks results in a system with strong self-explainability properties.

Abstract

It has been demonstrated in various contexts that monotonicity leads to better explainability in neural networks. However, not every function can be well approximated by a monotone neural network. We demonstrate that monotonicity can still be used in two ways to boost explainability. First, we use an adaptation of the decomposition of a trained ReLU network into two monotone and convex parts, thereby overcoming numerical obstacles from an inherent blowup of the weights in this procedure. Our proposed saliency methods -- SplitCAM and SplitLRP -- improve on state of the art results on both VGG16 and Resnet18 networks on ImageNet-S across all Quantus saliency metric categories. Second, we exhibit that training a model as the difference between two monotone neural networks results in a system with strong self-explainability properties.

Figures (8)

• Figure 1: Illustration of the network splitting procedure. A ReLU network $f$ is decomposed into two monotone and convex streams $g$ and $h$, such that $f = g - h$.
• Figure 2: Visualization of the monotone and convex split-streams (top) in comparison to a standard ReLU layer (bottom). The negative weight matrix transitions positive input to negative preactivation and vice versa, whereas the positive part of the weight transitions from positive to positive and negative to negative.
• Figure 3: Combined visualization of the inverted-input DIC gradients. The $g$-model gradient (red) focuses on present image parts, while the $h$-model gradient (blue) focuses on counterfactual features. On the left: the original digit.
• Figure 4: Saliency map comparison for VGG16 on our custom ImageNet-S test split. Each row shows the predicted class (with confidence), original image, and saliency maps from different methods.
• Figure 5: Concept visualization of the imaginative polytopes corresponding to VGG's dragonfly class output neuron. The polytope's vertices represent the $g$-stream and $h$-stream gradients respectively, revealing learned visual prototypes.

Theorems & Definitions (24)

• Theorem 1: Correctness
• Theorem 2
• proof : Proof of Theorem \ref{['thm:monotonicity_positive_gradients']}
• Theorem 3
• proof : Proof of Theorem \ref{['thm:monotone_maxout_main']}
• Theorem 4
• proof : Proof of Theorem \ref{['thm:main']}
• Claim 1: Correctness
• proof : Proof of Claim \ref{['claim:main_correct']}
• Theorem 5