A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks

TL;DR

Addressing the interpretability gap in CNNs, the paper proposes a unifying taxonomy that partitions feature-visualization methods into Input Modification, Deconvolutional, and Input Reconstruction categories, clarifying goals and algorithms. It also introduces the FeatureVis library built on MatConvNet, providing open-source implementations of methods from all three classes to facilitate experimentation and cross-method comparison. Through qualitative comparisons and architectural analyses, the work demonstrates how visualizations reveal learned intermediate representations and help explain differences in network performance. By standardizing terminology and providing tooling, the paper offers a practical foundation for interpretability research and future method development.

Abstract

Over the last decade, Convolutional Neural Networks (CNN) saw a tremendous surge in performance. However, understanding what a network has learned still proves to be a challenging task. To remedy this unsatisfactory situation, a number of groups have recently proposed different methods to visualize the learned models. In this work we suggest a general taxonomy to classify and compare these methods, subdividing the literature into three main categories and providing researchers with a terminology to base their works on. Furthermore, we introduce the FeatureVis library for MatConvNet: an extendable, easy to use open source library for visualizing CNNs. It contains implementations from each of the three main classes of visualization methods and serves as a useful tool for an enhanced understanding of the features learned by intermediate layers, as well as for the analysis of why a network might fail for certain examples.

Figures (4)

• Figure 1: Influences between authors of papers on feature visualization for CNNs. Dashed lines indicate that one author cited another, while solid lines indicate that one author build upon and extended the work of another. The ordering is top down by publication year.
• Figure 2: Different ways in which the pass through a ReLU layer affects contribution values for the Deconvnet method, Backpropagation, and Guided Backpropagation. The forward pass through the ReLU layer is shown for comparison.
• Figure 3: Comparing different visualization methods for two images. Both images where classified correctly with high confidence by the network. The image classes are: indigo bunting, indigo finch, indigo bird, Passerina cyanea and sax, saxophone. Visualizations for the respective image class using a VGG-16 network and an epsilon value of 0.001 for the Relevance Propagation method.
• Figure 4: Comparative visualizations of different networks for two images. Both images where classified correctly with high confidence by all networks. Visualizations for the respective image class using Guided Backpropagation. The image classes are: wombat and space shuttle.