Complete Neural Networks for Complete Euclidean Graphs

TL;DR

The paper tackles the problem of obtaining complete, permutation- and rigid-motion-invariant representations for 3D point clouds with polynomial complexity. It develops Euclidean WL tests by applying WL-style refinements to the point-cloud Gram matrix, proving completeness for two variants: $2$-SEWL and Vanilla $3$-EWL in $\mathbb{R}^3$, and showing that two iterations of $1$-EWL separate almost all point clouds. It further shows how to realize these tests with continuous, differentiable GNNs using multiset injective embeddings and intrinsic-dimension-aware design, culminating in the $2$-SEWLnet architecture. Synthetic experiments demonstrate the separation power on highly symmetric point clouds and compare to existing models, supporting the theoretical claims. Together, these results provide a principled route to universally distinguishing point clouds with architectures that remain tractable for practical learning tasks.

Abstract

Neural networks for point clouds, which respect their natural invariance to permutation and rigid motion, have enjoyed recent success in modeling geometric phenomena, from molecular dynamics to recommender systems. Yet, to date, no model with polynomial complexity is known to be complete, that is, able to distinguish between any pair of non-isomorphic point clouds. We fill this theoretical gap by showing that point clouds can be completely determined, up to permutation and rigid motion, by applying the 3-WL graph isomorphism test to the point cloud's centralized Gram matrix. Moreover, we formulate an Euclidean variant of the 2-WL test and show that it is also sufficient to achieve completeness. We then show how our complete Euclidean WL tests can be simulated by an Euclidean graph neural network of moderate size and demonstrate their separation capability on highly symmetrical point clouds.

Figures (3)

• Figure 1: Distance matrices (Left), geometric degree histogram (Right) of pairs of point clouds. The generic pair is a randomly sampled pair of point clouds. Notice each of the nodes in each of the clouds has a distinct geometric degree. The Hard pair exhibits a distinct geometric degree for each node, but only within each point cloud, that is the pair shares an identical geometric degree histogram. The Harder example is a pair of point clouds with identical geometric degree histogram, and each point cloud is comprised of three pairs of points, with each pair having an identical geometric degree. Examples from pozdnyakov2022incompleteness and pozdnyakov2020incompleteness.
• Figure 2: A plot of a Gaussian distribution centered at $x\in \mathbb{R}$, depicting a target function is shown in blue. In red, a schematic plot of how a Lipschitz continuous function that does not distinguish $x$ from $y$ would model the target function.
• Figure 3: The exponential growth in the dimension that would result from only considering the ambient feature dimension can be avoided by exploiting the constant intrinsic dimension.

Theorems & Definitions (21)

• Definition 1: Separating Invariant
• Theorem 1
• Theorem 1
• proof
• Theorem 2
• proof : Proof idea
• Theorem 1: dym2023low
• Theorem 2
• Theorem 1: Separation Implies Universality
• proof