On Universality Classes of Equivariant Networks
Marco Pacini, Gabriele Santin, Bruno Lepri, Shubhendu Trivedi
TL;DR
The paper tackles the question of when equivariant neural networks are universally expressive beyond their ability to separate inputs under symmetry. By formulating universality classes for shallow invariant networks and tying universality to differential constraints induced by basis maps, it shows that separation power alone does not determine approximation power, and that depth and the symmetry group's structure (e.g., normal subgroups) crucially influence universality. It provides concrete results, including a negative finding that certain shallow architectures can share maximal separation yet fail to be universal, and a positive pathway where normal-subgroup-based representations achieve separation-constrained universality. The work highlights a nuanced view of expressivity in symmetry-constrained models and lays groundwork for extending the framework to deeper networks and broader group actions.
Abstract
Equivariant neural networks provide a principled framework for incorporating symmetry into learning architectures and have been extensively analyzed through the lens of their separation power, that is, the ability to distinguish inputs modulo symmetry. This notion plays a central role in settings such as graph learning, where it is often formalized via the Weisfeiler-Leman hierarchy. In contrast, the universality of equivariant models-their capacity to approximate target functions-remains comparatively underexplored. In this work, we investigate the approximation power of equivariant neural networks beyond separation constraints. We show that separation power does not fully capture expressivity: models with identical separation power may differ in their approximation ability. To demonstrate this, we characterize the universality classes of shallow invariant networks, providing a general framework for understanding which functions these architectures can approximate. Since equivariant models reduce to invariant ones under projection, this analysis yields sufficient conditions under which shallow equivariant networks fail to be universal. Conversely, we identify settings where shallow models do achieve separation-constrained universality. These positive results, however, depend critically on structural properties of the symmetry group, such as the existence of adequate normal subgroups, which may not hold in important cases like permutation symmetry.