Understanding the Role of Pathways in a Deep Neural Network

TL;DR

This paper introduces diffusion pathways to interpret deep CNNs by tracing how individual input pixels propagate through the network to reach key feature maps. It presents an algorithm using diffusion kernels rotated by 180 degrees and masking strategies to extract pixel-level pathways and assemble portion-hot representations across a VGG-16 classifier trained on CIFAR, MNIST, and M2NIST. The findings show that a small set of large pathways per pixel cross the important maps, that pathway patterns are category-consistent in early layers and distinguishable across categories, and that diffusion pathways illuminate adversarial attacks, occlusion, and object movement. Collectively, the pathway framework provides a fine-grained, part-based view of internal representations, offering practical insights for explainability and robustness in DNNs and potentially shedding light on BNN-like information processing.

Abstract

Deep neural networks have demonstrated superior performance in artificial intelligence applications, but the opaqueness of their inner working mechanism is one major drawback in their application. The prevailing unit-based interpretation is a statistical observation of stimulus-response data, which fails to show a detailed internal process of inherent mechanisms of neural networks. In this work, we analyze a convolutional neural network (CNN) trained in the classification task and present an algorithm to extract the diffusion pathways of individual pixels to identify the locations of pixels in an input image associated with object classes. The pathways allow us to test the causal components which are important for classification and the pathway-based representations are clearly distinguishable between categories. We find that the few largest pathways of an individual pixel from an image tend to cross the feature maps in each layer that is important for classification. And the large pathways of images of the same category are more consistent in their trends than those of different categories. We also apply the pathways to understanding adversarial attacks, object completion, and movement perception. Further, the total number of pathways on feature maps in all layers can clearly discriminate the original, deformed, and target samples.

Figures (7)

• Figure 1: The diffusion pathways of an individual pixel and of an image within the VGG-16 CNN, which was trained using the CIFAR and MNIST datasets, and the comparison between large pathways and important feature maps for classification in each layer. (a) The VGG-16 CNN consists of 13 convolutional layers (conv1_1 through conv5_3) and 5 max pooling layers (maxpl1 through maxpl5), followed by 3 fully connected layers. (b) Diffusion pathways of an individual pixel, with each showing the first three largest pathway cross-sections in the feature map in each of the 18 layers of conv1_1 through maxpl5, from left to right, which are respectively marked by L0, L1, …, L17, and the same below. Diffusion pathways of an image randomly selected from the CIFAR dataset (c) and the MNIST dataset (d), with each showing the first three largest pathway cross-sections. The original image is in the first position. The distribution comparison between large pathway cross-sections (green) without channel mask constraints and important feature maps for classification (red) in each layer for a few images from the CIFAR dataset (e) and the MNIST dataset (f), and the Y-axis indicates the normalized intensity.
• Figure 2: The diffusion pathways within the VGG-16 CNN, which was trained using the CIFAR and MNIST datasets, and the comparison between small pathways and least important feature maps for classification in each layer. Pathways of an image randomly selected from the CIFAR dataset (a) and the MNIST dataset (c): each showing the three smallest pathways in each of the 18 layers of 13 convolutional layers and 5 pooling layers. Diffusion pathways of an image randomly selected from the CIFAR dataset (b) and the MNIST dataset (d), with each showing the pathways at the three least important feature map for classification. The distribution comparison between small pathways without channel mask constraints and least important feature maps for classification in each layer for images from the CIFAR dataset (e) and the MNIST dataset (f).
• Figure 3: Part pathways and main pathways in each layer. The original image is in the first row, the puzzle made up of parts is in the second row, and the parts are in the third (the largest, i.e., the main part) to the seventh row, in descending order by area. The horizontal axis indicates the 18 layers, from left to right. An image randomly selected from the CIFAR dataset (a), from the MNIST dataset (c), and from the M2NIST dataset (d) is segmented into five parts from top to bottom by the large pathways in each layer. (b) Nonoverlapping parts for the same image as in (a).
• Figure 4: The main pathways and the portion-hot representations along the 18 layers for intra- and intercategories under accurate classification. (a) The scatter of parts (spots) and the main pathways (plots) of 6 images from the same category '6' randomly selected from the CIFAR dataset. The Y-axis denotes the part index (feature map index), and the X-axis denotes the 18 layers. The spot size indicates the area ratio of parts to the whole image. (b) The scatter of parts (spots) and the main pathways (plots) of 10 images from 10 categories randomly selected from CIFAR dataset. (c) The scatter of parts (spots) and the main pathways (plots) of 8 images from the same category '5' of the MNIST dataset. (d) The scatter of parts (spots) and the main pathways (plots) of 10 images from 10 categories of the MNIST dataset. (e) The scatter of parts (spots) and the main pathways (plots) of 10 multilabel images from the M2NIST dataset. (f) The centers of the portion-hot representations (5696 dimensions) of each category randomly selected from the CIFAR dataset and their average center (‘cat.all’). (g) The saliency maps from the pathway-based method (upper) and the Grad-CAM method (lower). (h) The common parts within the 6 images same as in (a) corresponding to the pathway sections in feature maps 127 of layer 3 (top left), 26 of layer 5 (top right), 46 of layer 6 (lower left), and 94 of layer 7 (lower right).
• Figure 5: Understanding the adversarial phenomenon through the part pathways and the portion-hot representations. (a) The three largest parts at each of the 18 layers of a group of three images, from top to bottom: an original image, an adversarial version of the original image, and a target image. Different colors mark the three parts at each layer. The three initial images are in the first column. The part indexes are at the top of each subplot. (b) The portion-hot representations of the group of three images: the original (red), the adversarial (blue), and the target (yellow). The X-axis indicates the 5696 feature maps of the 18 layers, and the Y-axis indicates the part area ratio to image. The marker size also denotes the area ratio of a part. (c) The distances of the portion-hot representation of 400 groups randomly selected from the CIFAR dataset and generated by the FGSM: the distances between the original and the adversarial (red), between the original and the target (blue), and between the adversarial and the target (yellow). The X-axis indicates the group number, and the Y-axis indicates the distance of the portion-hot representation. (d) The index alignment for the three images of the feature maps that are important for classification at the upper end and of the large part pathways at the lower end along the 18 layers, and the Y-axis indicates the three images. Note: To highlight the distribution of each layer, the layers are set to be equally spaced along the X-axis.