Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks
Ashwin Samudre, Mircea Petrache, Brian D. Nord, Shubhendu Trivedi
TL;DR
This work addresses the challenge of rigid equivariance limiting performance in real-world tasks by introducing Group Matrices (GMs), a framework that unifies structured matrix design with group convolutions to create approximately equivariant neural networks over general discrete groups. By connecting GM theory with low displacement rank concepts, the authors develop GMConv and GMPool layers that realize group-aware convolutions and pooling while enabling controlled deviations from exact symmetry via learnable, low-rank perturbations. The approach yields parameter-efficient models that match or exceed the performance of existing approximately equivariant and structured-matrix methods across dynamics prediction and image classification benchmarks, often using orders of magnitude fewer parameters. The work further shows that the GM formalism naturally extends to homogeneous spaces and infinite discrete groups, offering a scalable path toward broader symmetry-informed architectures and potential steerability in future research.
Abstract
There has been much recent interest in designing neural networks (NNs) with relaxed equivariance, which interpolate between exact equivariance and full flexibility for consistent performance gains. In a separate line of work, structured parameter matrices with low displacement rank (LDR) -- which permit fast function and gradient evaluation -- have been used to create compact NNs, though primarily benefiting classical convolutional neural networks (CNNs). In this work, we propose a framework based on symmetry-based structured matrices to build approximately equivariant NNs with fewer parameters. Our approach unifies the aforementioned areas using Group Matrices (GMs), a forgotten precursor to the modern notion of regular representations of finite groups. GMs allow the design of structured matrices similar to LDR matrices, which can generalize all the elementary operations of a CNN from cyclic groups to arbitrary finite groups. We show GMs can also generalize classical LDR theory to general discrete groups, enabling a natural formalism for approximate equivariance. We test GM-based architectures on various tasks with relaxed symmetry and find that our framework performs competitively with approximately equivariant NNs and other structured matrix-based methods, often with one to two orders of magnitude fewer parameters.