HodgeFormer: Transformers for Learnable Operators on Triangular Meshes through Data-Driven Hodge Matrices

TL;DR

The paper introduces HodgeFormer, a Transformer variant that learns data-driven Hodge Laplacians on triangular meshes by parameterizing Hodge star operators via multi-head attention, thereby performing updates on vertices, edges, and faces without eigen-decomposition. Leveraging DEC-inspired operators and sparse attention, the model achieves competitive mesh classification and segmentation results while avoiding spectral preprocessing. The approach reduces reliance on expensive spectral features, offering an efficient, end-to-end architecture with O(n^{1.5}d) complexity and strong robustness to mesh perturbations. Overall, HodgeFormer advances mesh understanding by unifying discrete exterior calculus with attention mechanisms, enabling scalable, non-spectral geometric deep learning.

Abstract

Currently, prominent Transformer architectures applied on graphs and meshes for shape analysis tasks employ traditional attention layers that heavily utilize spectral features requiring costly eigenvalue decomposition-based methods. To encode the mesh structure, these methods derive positional embeddings, that heavily rely on eigenvalue decomposition based operations, e.g. on the Laplacian matrix, or on heat-kernel signatures, which are then concatenated to the input features. This paper proposes a novel approach inspired by the explicit construction of the Hodge Laplacian operator in Discrete Exterior Calculus as a product of discrete Hodge operators and exterior derivatives, i.e. $(L := \star_0^{-1} d_0^T \star_1 d_0)$. We adjust the Transformer architecture in a novel deep learning layer that utilizes the multi-head attention mechanism to approximate Hodge matrices $\star_0$, $\star_1$ and $\star_2$ and learn families of discrete operators $L$ that act on mesh vertices, edges and faces. Our approach results in a computationally-efficient architecture that achieves comparable performance in mesh segmentation and classification tasks, through a direct learning framework, while eliminating the need for costly eigenvalue decomposition operations or complex preprocessing operations.

Figures (13)

• Figure 1: The Hodge Laplacian applied on a 0-form, i.e. values on vertices, as a sequence of discrete exterior derivative and Hodge Star operators.
• Figure 2: Overview of a HodgeFormer layer operating on vertex, edge and face features via the Multi-Head Hodge Attention mechanism. The layer can be configured so that different mesh elements or combinations of them will be updated.
• Figure 3: Multi-head Hodge Attention applied on latent vertex features $x_v$. The multi-head attention mechanism learns data-driven Hodge Star matrices $\star_0^{-1}$ and $\star_1$. Layer Norm (LN) is applied on $Q$, $K$ matrices to increase training stability.
• Figure 4: Overview of a deep learning architecture with combined HodgeFormer and vanilla Transformer layers operating on an input triangular mesh.
• Figure 5: Mesh segmentation results on the Human Body test set.