Interpreting the Residual Stream of ResNet18

TL;DR

Problem: Understanding how DNNs compute visual representations remains incomplete; this work targets the residual stream of ResNet18 to gain mechanistic insight. Approach: It uses feature visualizations to examine center-neuron activations, defines a mix ratio $M_c = O_c(I)/O_c(B)$, and applies scale-invariance criteria built from scale operators $S$ and $S^{-1}$, plus a scale metric $SM_c$, to identify scale-invariant residual features. Findings: The residual stream shows a skip-overwrite-mix spectrum; some early- to mid-block channels exhibit scale-invariant behavior; weight and correlation analyses suggest bypass by lower-magnitude block outputs and mixed evidence for inhibitory interactions. Significance: Demonstrates a concrete mechanistic interpretation of residual streams, supporting scale-equivariance ideas and suggesting potential universality across architectures and even biological visual systems.

Abstract

A mechanistic understanding of the computations learned by deep neural networks (DNNs) is far from complete. In the domain of visual object recognition, prior research has illuminated inner workings of InceptionV1, but DNNs with different architectures have remained largely unexplored. This work investigates ResNet18 with a particular focus on its residual stream, an architectural mechanism which InceptionV1 lacks. We observe that for a given block, channel features of the stream are updated along a spectrum: either the input feature skips to the output, the block feature overwrites the output, or the output is some mixture between the input and block features. Furthermore, we show that many residual stream channels compute scale invariant representations through a mixture of the input's smaller-scale feature with the block's larger-scale feature. This not only mounts evidence for the universality of scale equivariance, but also presents how the residual stream further implements scale invariance. Collectively, our results begin an interpretation of the residual stream in visual object recognition, finding it to be a flexible feature manager and a medium to build scale invariant representations.

Figures (8)

• Figure 1: Each grid contains feature visualizations of the channel's center neuron, the entire channel, and the top 9 center neuron activating natural images from the ImageNet validation set 5206848. All channel indices are zero-indexed. A: Exemplary overwrite (38), mix (19), and skip (25) channels from output 1.1 (1.0 input + 1.1.bn2 block). Note the presence or lack of overlap in top vals between Output and Input and/or Block. B: Exemplary scale invariant channels (178, 215, 25) from output 3.1.
• Figure 2: Histograms of mix ratios for all inspected blocks of ResNet18, starting at 1.1 through 4.1. Bins are from 0 to 5 at 0.25 intervals. Close to 1 is a mixture of input and block features, below 1 indicates an overwrite behavior, above 1 indicates a skip behavior.
• Figure 3: Weight magnitudes with respect to mix ratio of second batch normalization (top row) and the second convolutional layer (bottom row, mean taken). Spearman rank correlations and p-values displayed for each block, which show that channels with more skip-like behavior have smaller weight magnitudes.
• Figure 4: Top Row: Activations of second batch norm (pre-sum block output) with respect to mix ratio, with activation means for skip (mix > 1, blue) and overwrite (mix < 1, red) channels. Bottom Row: Spearman rank correlations of Block activations with respect to Input activations for ImageNet validation images which positively activate Input, plotted against mix ratio. ANOVA Fs and p-values between skip and overwrite activations displayed for each block.
• Figure 5: Percent of channels that meet all three scale invariance criteria (purple), and those with the mix ratio criterion omitted (yellow).