Volume Tracking Based Reference Mesh Extraction for Time-Varying Mesh Compression

TL;DR

Time-Varying Meshes (TVMs) with changing topology introduce substantial storage and processing challenges for dynamic surfaces. The paper presents a volume-tracking based reference mesh extraction pipeline that combines ARAP volume tracking to place centers, Multidimensional Scaling (MDS) to derive 3D reference centers, Radial Basis Function (RBF) mapping to align frame vertices, and Poisson surface reconstruction to generate a self-contact-free reference mesh, followed by ARAP-based re-meshing for per-frame deformation. Key contributions include a robust method for generating a reference mesh suitable for surface correspondence and deformation across TVMs, an optimization step that improves cross-frame alignment within groups of frames (GoFs), and an enhanced keypoint-driven deformation mechanism. This approach enables more reliable surface correspondence and deformation-based TVM processing, with potential practical impact on efficient TVM compression and dynamic surface reconstruction.

Abstract

Time-Varying meshes (TVMs), characterized by their varying connectivity and number of vertices, hold significant potential in immersive media and other various applications. However, their practical utilization is challenging due to their time-varying features and large file sizes. Creating a reference mesh that contains the most essential information is a promising approach to utilizing shared information within TVMs to reduce storage and transmission costs. We propose a novel method that employs volume tracking to extract reference meshes. First, we adopt as-rigid-as-possible (ARAP) volume tracking on TVMs to get the volume centers for each mesh. Then, we use multidimensional scaling (MDS) to get reference centers that ensure the reference mesh avoids self-contact regions. Finally, we map the vertices of the meshes to reference centers and extract the reference mesh. Our approach offers a feasible solution for extracting reference meshes that can serve multiple purposes such as establishing surface correspondence, deforming the reference mesh to different shapes for I-frame based mesh compression, or defining the global shape of the TVMs.

Figures (7)

• Figure 1: Distortions caused by self-contact region.
• Figure 2: The simplified workflow of our proposed volume tracking-based reference mesh extraction method.
• Figure 3: Visual comparison of key points matching process. The yellow points set represents the key point on the palm and its surrounding neighbors, while the blue points set represents the key point on the back of the hand and its surrounding neighbors. Although these two key points have similar surrounding neighbors, they do not match. In (a), the two key points are mismatched, whereas in (b), our approach successfully identifies their differences.
• Figure 4: Visual comparison of matched key points. (a) displays the point matching results from keypoint matching, while (b) presents the matching results using our method. The matched key point pairs are connected by red lines. Some connections in (a) on the arm are mismatched, e.g. the key point on the right hand was incorrectly matched to the arm. Compared to (a), the connections in (b) are more accurately matched.
• Figure 5: Comparison of Hausdorff Distance for different center numbers across various GoFs. The plots illustrate the performance of three TVMs: (a) Levi, (b) Dancer, and (c) Basketball player. Each graph shows the Hausdorff Distance for three different center numbers: 500 (blue circle), 1000 (orange square), and 1500 (green triangle). The Hausdorff distance measures the geometry distance between vertices, making it sensitive to the scale of mesh sequences. Hence, different mesh sequences will have varying ranges of Hausdorff distances.