Feature Visualization in 3D Convolutional Neural Networks

TL;DR

This work tackles the interpretability of 3D convolutional kernels in video models by introducing a data-driven decomposition that separates texture and motion in the kernel's optimal activation input. It first optimizes a video input $V=\{V_t\}_{t=1}^T$ to maximally activate a target kernel, then decomposes $V$ into a static texture component $I$ and per-frame deformation fields $D_t$ through a warping operator $\mathcal{W}(I,D_t)$. A two-stage optimization is proposed: Stage 1 pixel-domain activation maximization and Stage 2 decomposition with a composite loss $\mathcal{L}_{s2}=\mathcal{L}_{recon}+\mathcal{L}_{smooth_D}+\mathcal{L}_{smooth_I}+\mathcal{L}_{static}$. Across models such as I3D, C3D, and 3D VQ-VAE, the method yields interpretable visualizations that reveal texture preferences and motion patterns, showing increasing spatiotemporal complexity with depth and providing actionable insights into 3D kernel representations.

Abstract

Understanding the computations of convolutional neural networks requires effective visualization of their kernels. While maximal activation methods have proven successful in highlighting the preferred features of 2D convolutional kernels, directly applying these techniques to 3D convolutions often leads to uninterpretable results due to the higher dimensionality and complexity of 3D features. To address this challenge, we propose a novel visualization approach for 3D convolutional kernels that disentangles their texture and motion preferences. Our method begins with a data-driven decomposition of the optimal input that maximally activates a given kernel. We then introduce a two-stage optimization strategy to extract distinct texture and motion components from this input. Applying our approach to visualize kernels at various depths of several pre-trained models, we find that the resulting visualizations--particularly those capturing motion--clearly reveal the preferred dynamic patterns encoded by 3D kernels. These results demonstrate the effectiveness of our method in providing interpretable insights into 3D convolutional operations. Code is available at https://github.com/YatangLiLab/3DKernelVisualizer.

Figures (6)

• Figure 1: Overview of visualization process.
• Figure 2: Illustration of the single frame reconstruction from decomposition factors: (a) Static factor $I$, (b) Deformation field $D_t$ at time $t$, (c) Frame $V_t$ at time t synthesized from (a) and (b).
• Figure 3: Visualization examples optimized in different domains. (a-c) are optimized in the Fourier domain, while (d-f) are in the pixel domain. Each column shows kernels at different layers. (a, d) are from I3D stage 1, (b, e) are from stage 3, and (c, f) are from stage 5.
• Figure 4: Ablation study on the loss function, using a kernel from I3D stage 4. (a) Color coding of the deformation field, where hue denotes the motion direction and value denotes the motion amplitude of each pixel. (b) The optimal input in the pixel domain. (c-f) Learnable components under different loss functions in the second training stage, where the first column is the static factor, and the last 3 columns are deformation fields at different times: (c) only includes reconstruction loss. (d) adds deformation TV loss, (e) adds static TV loss, (f) uses the full loss. (g) Reconstructed video with full loss.
• Figure 5: Ablation study on the optimization strategy. The first row shows pixel-domain visualizations from the two-stage method; the second row shows results from a single-stage approach.