Revisiting Transformation Invariant Geometric Deep Learning: An Initial Representation Perspective
Ziwei Zhang, Xin Wang, Zeyang Zhang, Peng Cui, Wenwu Zhu
TL;DR
This work tackles transformation invariance in geometric deep learning, showing that a distance-preserving, transformation-invariant initial representation suffices to guarantee invariance when fused with ordinary neural backbones. The authors propose TinvNet, a simple MDS-based plug-in that yields an invariant embedding fed into any network, with formal guarantees under similarity transformations and practical validation on point-cloud tasks and combinatorial optimization problems. The key contributions include a theoretical framework for invariance, a concrete MDS-based construction, empirical evidence of robustness across rotations, translations, reflections, and scaling, and an extension path toward equivariance. The approach offers a lightweight, flexible baseline that can be combined with existing architectures to improve transformation robustness in geometric learning tasks.
Abstract
Deep neural networks have achieved great success in the last decade. When designing neural networks to handle the ubiquitous geometric data such as point clouds and graphs, it is critical that the model can maintain invariance towards various transformations such as translation, rotation, and scaling. Most existing graph neural network (GNN) approaches can only maintain permutation-invariance, failing to guarantee invariance with respect to other transformations. Besides GNNs, other works design sophisticated transformation-invariant layers, which are computationally expensive and difficult to be extended. In this paper, we revisit why general neural networks cannot maintain transformation invariance. Our findings show that transformation-invariant and distance-preserving initial point representations are sufficient to achieve transformation invariance rather than needing sophisticated neural layer designs. Motivated by these findings, we propose Transformation Invariant Neural Networks (TinvNN), a straightforward and general plug-in for geometric data. Specifically, we realize transformation invariant and distance-preserving initial point representations by modifying multi-dimensional scaling and feed the representations into existing neural networks. We prove that TinvNN can strictly guarantee transformation invariance, being general and flexible enough to be combined with the existing neural networks. Extensive experimental results on point cloud analysis and combinatorial optimization demonstrate the effectiveness and general applicability of our method. We also extend our method into equivariance cases. Based on the results, we advocate that TinvNN should be considered as an essential baseline for further studies of transformation-invariant geometric deep learning.