Physics-Driven Local-Whole Elastic Deformation Modeling for Point Cloud Representation Learning

TL;DR

The paper addresses the limitation of purely data-driven point-cloud representations in capturing how local geometry deforms and propagates to the whole shape under external forces. It introduces a dual-task framework that jointly learns an implicit field for the global shape and a physics-informed deformation predictor based on a tetrahedral mesh, supervised by data-fidelity and force-equilibrium losses. Key contributions include the integration of static linear elasticity as a lightweight prior, a tetrahedral-mesh-based physics module, and comprehensive experiments showing improved classification and segmentation, particularly on real-world data, along with enhanced interpretability of local-to-global relationships. This physics-grounded approach yields more robust and explanations-rich point-cloud representations with stronger generalization for downstream 3D understanding tasks.

Abstract

Existing point cloud representation learning methods primarily rely on data-driven strategies to extract geometric information from large amounts of scattered data. However, most methods focus solely on the spatial distribution features of point clouds while overlooking the relationship between local information and the whole structure, which limits the accuracy of point cloud representation. Local information reflect the fine-grained variations of an object, while the whole structure is determined by the interaction and combination of these local features, collectively defining the object's shape. In real-world, objects undergo deformation under external forces, and this deformation gradually affects the whole structure through the propagation of forces from local regions, thereby altering the object's geometric features. Therefore, appropriately introducing a physics-driven mechanism to capture the topological relationships between local parts and the whole object can effectively mitigate for the limitations of data-driven point cloud methods in structural modeling, and enhance the generalization and interpretability of point cloud representations for downstream tasks such as understanding and recognition. Inspired by this, we incorporate a physics-driven mechanism into the data-driven method to learn fine-grained features in point clouds and model the structural relationship between local regions and the whole shape. Specifically, we design a dual-task encoder-decoder framework that combines the geometric modeling capability of data-driven implicit fields with physics-driven elastic deformation. Through the integration of physics-based loss functions, the framework is guided to predict localized deformation and explicitly capture the correspondence between local structural changes and whole shape variations.

Figures (7)

• Figure 1: (a) Intra-class deformation and force distribution under an external force $f$ for Lamp A and B (colors indicate force magnitude; highlighted regions show significant deformation). (b) Inter-class clustering visualization in the embedding space across multiple categories, comparing models without and with Physics-Informed (PI) modeling. Note that (a) and (b) correspond to different tasks and the color encodings are not shared.
• Figure 2: The overall architecture of the proposed method is illustrated. It consists of two modules: the implicit feature learning (IFL) module, which represents the point cloud as a continuous surface, helping the network better learn the whole shape features of the object; and the physics information awareness (PIA) module, which learns the relationship between local features and whole geometry through the elastic deformation of the object, enhancing the model’s ability to learn local details.
• Figure 3: The point cloud is first converted into a coarse mesh form through Delaunay triangulation. Then, we removed the larger tetrahedral meshes, further refining them into the final tetrahedral representation.
• Figure 4: Feature-space visualization (t-SNE) of learned point-cloud embeddings on ShapeNet with representative baselines. (a) Ours with PointNet backbone, (b) Ours with DGCNN backbone, (c) STRL (DGCNN), and (d) CrossPoint (DGCNN). Colors indicate object categories.
• Figure 5: Semantic segmentation results on S3DIS (DGCNN as backbone). Different colors indicate different objects. From left to right are the top view of the ground truth, the predicted top view, the front view, and the side view.