InfoGNN: End-to-end deep learning on mesh via graph neural networks

TL;DR

InfoGNN reframes 3D mesh learning as end-to-end graph learning by treating meshes as graphs and introducing InfoConv and InfoMP to fuse static geometric attributes, dynamic local details, and global shape cues. The approach eliminates mesh-specific constraints, enabling efficient and scalable learning for mesh classification and segmentation, with dynamic global features providing consistent performance gains. Extensive experiments on ShapeNetCore, ModelNet40, and human-body datasets, coupled with thorough ablations, demonstrate competitive accuracy and robustness, highlighting the practicality of end-to-end mesh GNNs for real-world 3D vision tasks.

Abstract

3D models are widely used in various industries, and mesh data has become an indispensable part of 3D modeling because of its unique advantages. Mesh data can provide an intuitive and practical expression of rich 3D information. However, its disordered, irregular data structure and complex surface information make it challenging to apply with deep learning models directly. Traditional mesh data processing methods often rely on mesh models with many limitations, such as manifold, which restrict their application scopes in reality and do not fully utilize the advantages of mesh models. This paper proposes a novel end-to-end framework for addressing the challenges associated with deep learning in mesh models centered around graph neural networks (GNN) and is titled InfoGNN. InfoGNN treats the mesh model as a graph, which enables it to handle irregular mesh data efficiently. Moreover, we propose InfoConv and InfoMP modules, which utilize the position information of the points and fully use the static information such as face normals, dihedral angles, and dynamic global feature information to fully use all kinds of data. In addition, InfoGNN is an end-to-end framework, and we simplify the network design to make it more efficient, paving the way for efficient deep learning of complex 3D models. We conducted experiments on several publicly available datasets, and the results show that InfoGNN achieves excellent performance in mesh classification and segmentation tasks.

Figures (7)

• Figure 1: The input is a 3D mesh model. We preprocess the model, convert it into a unified representation and storage in the form of a graph, and extract and store simple and useful information. After that, we feed this graph data into the message-passing graph neural network InfoMP for feature aggregation and learning to output useful features. The numbers in parentheses represent the input and output dimensions. The hidden layer of InfoMP consists of multiple parts, which are generally twice as large as the input dimension. After 4 InfoMP layers with residual connections, the feature vectors of all nodes are obtained. Depending on the task, a suitable classification head can be chosen. Here, we use a simple multilayer perceptron (MLP) for classification. The numbers in parentheses below represent the input dimensions, hidden layers, and output layers of the MLP.
• Figure 2: Model information data composition. The graph data of each model contains six different types of information that can be categorized into node information and edge information. The node information includes the original point coordinate information inherent in each model. Information such as face center coordinates, face normal vector, and area can be calculated based on the original point coordinates. The edge information is constructed from the face center coordinates using the KNN algorithm, which has the advantages of time-saving and adjustability. Combining the face normal vector and edge information, the simple feature information of the edge, such as the dihedral angle, can be further calculated.
• Figure 3: Some samples from the human body dataset and their corresponding renderings of the segmentation results. As can be seen, our segmentation results are mostly accurate, but there are still some boundary divergences that we aim to improve in the future.
• Figure 4: Some samples from ShapeNetCore and their corresponding renderings of the segmentation results.It can be seen that this dataset segmentation quality is not high and there are some incorrect annotations, but our model still gives results that are more in line with human intuition. This suggests that our model learns better features and is able to effectively utilize the other correct annotations in the dataset for more rational classification.
• Figure 5: Some example shapes from ModelNet40 are used to demonstrate the classification capabilities of our network.