Stability Analysis of Equivariant Convolutional Representations Through The Lens of Equivariant Multi-layered CKNs

TL;DR

This work develops group-equivariant Convolutional Kernel Networks (CKNs) within the reproducing kernel Hilbert space (RKHS) framework to analyze the geometry and robustness of equivariant representations. It provides a detailed multilayer construction using patch extraction $P_k$, kernel mapping $M_k$, and pooling $A_k$, with norm-preserving, non-expansive kernel mappings and $G$-equivariance at each layer, yielding a hierarchical representation $\Phi_N$ for which a deformation stability bound is established: $||\Phi_N(L_\tau x) - \Phi_N(x)|| \le ( C_1 (1+N) ||\nabla\tau||_\infty + C_2/\sigma_N ||\tau||_\infty ) ||x||$. The paper also provides empirical evidence on deformation stability for SE(2) and SO(3) and outlines how to embed equivariant CNNs in RKHSs, linking RKHS norms to generalization bounds. Together, these results offer principled guidance for designing robust, symmetry-aware representations and point to future extensions to manifolds, gauge theories, and broader generalization analyses via RKHS-based methods.

Abstract

In this paper we construct and theoretically analyse group equivariant convolutional kernel networks (CKNs) which are useful in understanding the geometry of (equivariant) CNNs through the lens of reproducing kernel Hilbert spaces (RKHSs). We then proceed to study the stability analysis of such equiv-CKNs under the action of diffeomorphism and draw a connection with equiv-CNNs, where the goal is to analyse the geometry of inductive biases of equiv-CNNs through the lens of reproducing kernel Hilbert spaces (RKHSs). Traditional deep learning architectures, including CNNs, trained with sophisticated optimization algorithms is vulnerable to perturbations, including `adversarial examples'. Understanding the RKHS norm of such models through CKNs is useful in designing the appropriate architecture and can be useful in designing robust equivariant representation learning models.

Figures (2)

• Figure 1: A schematic diagram of $1$-layer of a CKN where one constructs $k$-th signal representation from the $k$$-1$-th one in a RKHS $\mathcal{H}_k$ through patch extraction, kernel mapping and pooling operators, as similarly shown in bietti2019group. Signal domain $\Omega = \mathbb{R}^d$ (in this figure $d=2$) on which locally compact group $G$ acts. One can construct a multilayered CKN by stacking these layers in a hierarchical manner and make the entire network equivariant by making each layers equivariant to the action of $G$.
• Figure 2: Stability analysis with equiv-CKNs and comparing with CKNs. The first row represent experiments with rotated MNIST with $G=SE(2)$, whereas the second column is experiments on rotated MNIST on sphere $S^2$ with $G=SO(3)$. We evaluate mean average distances while varying deformation scale $\alpha =\{ 0.1,0.5,1,2.5,5\}$, patch size $\kappa= \{ 2,5,8,10\}$ and scale of last pooling layer $h_k$, $\sigma_k=\{ 1,3,5,10\}$. For experiments with patch size and last pooling layer parameter $\sigma$, we keep $\alpha=1$ and choose RBF kernel with bandwidth $\{ 5,10\}$ first column, and exponential kernel, $k_{exp}(\langle x,x'\rangle) = exp(\langle x,x'\rangle -1)$ for our kernel mapping for the second column.

Theorems & Definitions (22)

• Lemma 2.1: Lemma 1, bietti2019group
• Theorem 2.2: Equivariance of a CKN
• proof
• Corollary 2.3: Equivariant convolutional kernels in RKHS
• Lemma 3.1
• proof
• Theorem 3.2: Stability bound
• Proposition 3.3
• proof
• Theorem 1.1