Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimization

TL;DR

Overall, MACO unlocks, for the first time, feature visualizations for large, state-of-the-art deep neural networks without resorting to any parametric prior image model.

Abstract

Feature visualization has gained substantial popularity, particularly after the influential work by Olah et al. in 2017, which established it as a crucial tool for explainability. However, its widespread adoption has been limited due to a reliance on tricks to generate interpretable images, and corresponding challenges in scaling it to deeper neural networks. Here, we describe MACO, a simple approach to address these shortcomings. The main idea is to generate images by optimizing the phase spectrum while keeping the magnitude constant to ensure that generated explanations lie in the space of natural images. Our approach yields significantly better results (both qualitatively and quantitatively) and unlocks efficient and interpretable feature visualizations for large state-of-the-art neural networks. We also show that our approach exhibits an attribution mechanism allowing us to augment feature visualizations with spatial importance. We validate our method on a novel benchmark for comparing feature visualization methods, and release its visualizations for all classes of the ImageNet dataset on https://serre-lab.github.io/Lens/. Overall, our approach unlocks, for the first time, feature visualizations for large, state-of-the-art deep neural networks without resorting to any parametric prior image model.

Figures (20)

• Figure 1: Comparison between feature visualization methods for "White Shark" classification.(Top) Standard Fourier preconditioning-based method for feature visualization olah2017feature. (Bottom) Proposed approach, MACO, which incorporates a Fourier spectrum magnitude constraint.
• Figure 2: Comparison between Fourier FV and natural image power spectrum. In (a), the power spectrum is averaged over $10$ different logits visualizations for each of the $1000$ classes of ImageNet. The visualizations are obtained using the Fourier FVFourier FV method to maximize the logits of a ViT network olah2017feature. In (b) the spectrum is averaged over all training images of the ImageNet dataset.
• Figure 3: Overview of the approach:(a) Current Fourier parameterization approaches optimize the entire spectrum (yellow arrow). (b) In contrast, the optimization flow in our approach (green arrows) goes from the network activation ($\bm{y}$) to the phase of the spectrum ($\bm{\varphi}$) of the input image ($\bm{x}$).
• Figure 4: (left) Logits and (right) internal representations of FlexiViT.MACO was used to maximize the activations of (left) logit units and (right) specific channels located in different blocks of the FlexViT (blocks 1, 2, 6 and 10 from left to right).
• Figure 5: Human causal understanding of model activations. We follow the experimental procedure introduced in zimmermann2021well to evaluate Olah and MACO visualizations on $3$ different networks. The control condition is when the participant did not see any feature visualization.

Theorems & Definitions (1)

• Definition 3.1: MACO