The Mechanics of CNN Filtering with Rectification

TL;DR

This work introduces elementary information mechanics to describe how CNN filters with rectification propagate information, using an even–odd decomposition that maps to a relativistic energy–momentum framework. It identifies three propagation modes—diffusion, vibration, translation—and demonstrates that the three lowest-frequency DCT components, $Σ$, $∇x$, and $∇y$, dominate network performance, capturing the majority of accuracy in standard architectures. Through DCT-based training experiments on ImageNet (VGG16, ResNet50) and CIFAR-100 (VGG16, ResNet20), it shows that these components, with light fine-tuning, nearly reproduce baseline results. The findings offer a new lens on CNN information flow, with potential implications for optimization, initialization, and architecture design, by linking filter mechanics to energy–momentum relations observed in physics.

Abstract

This paper proposes elementary information mechanics as a new model for understanding the mechanical properties of convolutional filtering with rectification, inspired by physical theories of special relativity and quantum mechanics. We consider kernels decomposed into orthogonal even and odd components. Even components cause image content to diffuse isotropically while preserving the center of mass, analogously to rest or potential energy with zero net momentum. Odd kernels cause directional displacement of the center of mass, analogously to kinetic energy with non-zero momentum. The speed of information displacement is linearly related to the ratio of odd vs total kernel energy. Even-Odd properties are analyzed in the spectral domain via the discrete cosine transform (DCT), where the structure of small convolutional filters (e.g. $3 \times 3$ pixels) is dominated by low-frequency bases, specifically the DC $Σ$ and gradient components $\nabla$, which define the fundamental modes of information propagation. To our knowledge, this is the first work demonstrating the link between information processing in generic CNNs and the energy-momentum relation, a cornerstone of modern relativistic physics.

Figures (26)

• Figure 1: Motivating example: training using filters limited to DCT component subsets reveals that the three lowest frequency components (labelled $\Sigma,\nabla_x, \nabla_y$) account the majority of baseline accuracy for Resnet and VGG16 models (94% and 92% respectively). Details are provided in \ref{['dct_experiments']}.
• Figure 2: The three most significant components of real linear decompositions resemble sum $\Sigma$ and gradient $\nabla_x, \nabla_y$ filters, including a) JPEG DCT coefficients wallace1992jpeg b) Natural image PCs olshausen1996emergence c) CNN PCs (VGG16) fukuzaki2022principal.
• Figure 3: a) The decomposition of an example 1D filter $F=[0,2]$ into even $f_e=[1,1]$ and odd $f_o=[-1,1]$ components based on left-right symmetry. b) The Pythagorean relationship between orthogonal even and odd components.
• Figure 4: Illustrating the geometry of a 2D filter. a) shows an example decomposition of a 2D filter $F$ into even $f_e$ and odd $f_o$ components. b) shows the Pythagorean relationship between the Euclidean magnitudes of orthogonal $f_e$ and $f_o$ components. c) illustrates 3D filter component space with a vertical even axis $f_e \approx \Sigma$ and a horizontal gradient plane $f_o \approx \{\nabla_x, \nabla_y\}$.
• Figure 5: The effect of repeated convolution+ReLU of an impulse test pattern with a $3\times3$ kernel mixing DC $\Sigma$ and gradient $\nabla_x$ components at three different ratios $\beta^2=\{0,.25,1\}$.

Theorems & Definitions (4)

• Definition 1: Even (Symmetric) Image
• Definition 2: Odd (Anti-Symmetric) Image
• Lemma 1: Orthogonality
• Definition 3: Energy