Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks

TL;DR

The paper tackles interpretability of deep neural networks by showing that individual neurons can be multifaceted rather than single-feature detectors. It introduces Multifaceted Feature Visualization (MFV), which identifies distinct facets of a neuron by clustering class exemplars in a low-dimensional space and initializing activation maximization from facet-specific mean images; it also adds center-biased regularization to produce coherent visuals. Results demonstrate that higher-layer neurons exhibit greater facet diversity, and MFV yields more natural colors and globally consistent facet visuals, thereby advancing the state of activation maximization. This approach enhances understanding of feature representations across layers and suggests practical avenues for more interpretable network design and visualization tools.

Abstract

We can better understand deep neural networks by identifying which features each of their neurons have learned to detect. To do so, researchers have created Deep Visualization techniques including activation maximization, which synthetically generates inputs (e.g. images) that maximally activate each neuron. A limitation of current techniques is that they assume each neuron detects only one type of feature, but we know that neurons can be multifaceted, in that they fire in response to many different types of features: for example, a grocery store class neuron must activate either for rows of produce or for a storefront. Previous activation maximization techniques constructed images without regard for the multiple different facets of a neuron, creating inappropriate mixes of colors, parts of objects, scales, orientations, etc. Here, we introduce an algorithm that explicitly uncovers the multiple facets of each neuron by producing a synthetic visualization of each of the types of images that activate a neuron. We also introduce regularization methods that produce state-of-the-art results in terms of the interpretability of images obtained by activation maximization. By separately synthesizing each type of image a neuron fires in response to, the visualizations have more appropriate colors and coherent global structure. Multifaceted feature visualization thus provides a clearer and more comprehensive description of the role of each neuron.

Figures (14)

• Figure 1: Top: Visualizations of 8 types of images (feature facets) that activate the same "grocery store" class neuron. Bottom: Example training set images that activate the same neuron, and resemble the corresponding synthetic image in the top panel.
• Figure 2: Progressive result of optimizing an image to activate the "milk can" neuron via the center-biased regularization method.
• Figure 3: Multifaceted visualization of $\mathsf{fc8}\xspace$ units uncovers interesting facets. We show 4 different facets for each neuron. In each pair of images, the bottom is the facet visualization that represents a cluster of images from the training set, and the top is the closest image to the visualization from the same cluster.
• Figure 4: Visualizing the different facets of a neuron that detects bell peppers. Diverse facets include a single, red bell pepper on a white background (1), multiple red peppers (5), yellow peppers (8), and green peppers on: the plant (4), a cutting board (6), or against a dark background (10). Center: training set Images from the bell pepper class are projected into two dimensions by t-SNE and clustered by $k$-means (see Sec. \ref{['sec:mfv']}). Sides: synthetic images generated by multifaceted feature visualization for the "bell pepper" class neuron for each of the 10 numbered facets. Best viewed electronically, in color, with zoom.
• Figure 5: Visualizing the different facets of a neuron that detects images in the "fishing reel" class. Diverse facets include reels on backgrounds that are: white (2), dark (3), ocean blue (7) or forest green (8); reels placed next to fish laying on grass (4), people fishing at sea (5), and a specific type of reel with holes in it (9). Each reconstruction is a facet visualization for a cluster of images in the "fishing reel" class. The image components are as described in Fig. \ref{['fig:tsne_bellpeper']}, except next to each facet visualization, we include the four images in each facet closest to the center of that facet cluster. Best viewed electronically with zoom.