Adaptive Sampling for Continuous Group Equivariant Neural Networks

TL;DR

An adaptive sampling approach is introduced that dynamically adjusts the sampling process to the symmetries in the data, reducing the number of required group samples and lowering the computational demands.

Abstract

Steerable networks, which process data with intrinsic symmetries, often use Fourier-based nonlinearities that require sampling from the entire group, leading to a need for discretization in continuous groups. As the number of samples increases, both performance and equivariance improve, yet this also leads to higher computational costs. To address this, we introduce an adaptive sampling approach that dynamically adjusts the sampling process to the symmetries in the data, reducing the number of required group samples and lowering the computational demands. We explore various implementations and their effects on model performance, equivariance, and computational efficiency. Our findings demonstrate improved model performance, and a marginal increase in memory efficiency.

Figures (13)

• Figure 1: Behaviour of sampled values from a translated feature when using a fixed and an adaptive grid. Left column: the continuous feature function as well as the location of the grid samples (in colors). Right column: the measured values at the grid locations. When the function is shifted, the sampled values on the fixed grid change; these samples can not accurately capture translations smaller than the grid resolution. Instead, the adaptive grid is translated together with the input function, ensuring the measured values are constant (compare columns of the same color in the first and third row). Information about the phase is preserved by the adaptive grid itself (note the colored grid points translates too).
• Figure 2: Discretized Inverse Fourier transform, where $\hat{f} = \bigoplus_\psi vec(\hat{f}(\psi)) \in \mathbb{R}^{F}$ and $A_i:= \rho(g_i)\hat{\delta} \in \mathbb{R}^F$. $N$ is the number of group samples, while $F$ is the size of the representation. Each row in $A$ corresponds to a single group sample, while each column corresponds to an irrep's matrix coefficients.
• Figure 3: Architecture for point cloud processing. In this architecture, blue blocks at the bottom represents the main branch which process the point clouds and the gray blocks at the top correspond to sampling branch which generates the sampling matrix $A$ and perform spatial downsampling accordingly. Although it is not illustrated in the figure, each convolutional block, which comprises the convolutional layer, batch normalization and the nonlinear layer, is followed by an equivariant MLP.
• Figure 4: Test accuracy on ModelNet10, with respect to the number of group samples by model.
• Figure 5: Test accuracies vs. memory cost by model. Results are computed on ModelNet10.