TagSplat: Topology-Aware Gaussian Splatting for Dynamic Mesh Modeling and Tracking

TL;DR

This work tackles the challenge of reconstructing topology-consistent dynamic meshes from multi-view video. It introduces a topology-aware Gaussian Splatting framework that attaches a Gaussian topology structure to enforce surface connectivity, with topology-preserving densification/pruning and temporal regularization to maintain coherence across frames, complemented by differentiable mesh rasterization. The approach yields topology-consistent Gaussian and mesh sequences and enables robust 3D keypoint tracking, outperforming baseline Gaussian-based methods on synthetic MIX-TAG and real TalkBody4D data. The method offers a practical, low-cost pathway for animation and model editing by unifying Gaussian representations with mesh topology, while acknowledging limitations in handling drastic topology changes and outlining directions for future work.

Abstract

Topology-consistent dynamic model sequences are essential for applications such as animation and model editing. However, existing 4D reconstruction methods face challenges in generating high-quality topology-consistent meshes. To address this, we propose a topology-aware dynamic reconstruction framework based on Gaussian Splatting. We introduce a Gaussian topological structure that explicitly encodes spatial connectivity. This structure enables topology-aware densification and pruning, preserving the manifold consistency of the Gaussian representation. Temporal regularization terms further ensure topological coherence over time, while differentiable mesh rasterization improves mesh quality. Experimental results demonstrate that our method reconstructs topology-consistent mesh sequences with significantly higher accuracy than existing approaches. Moreover, the resulting meshes enable precise 3D keypoint tracking. Project page: https://haza628.github.io/tagSplat/

Figures (15)

• Figure 1: Comparison of training results for the Worker object.
• Figure 2: Illustration of the Gaussian topology structure. Each Gaussian primitive is connected via manifold edges. Compared with the original 3D Gaussians, our structure constrains the relative positions and rotations of neighboring Gaussians. This method enables training Gaussian and mesh models with consistent topology.
• Figure 2: Comparison of training results for the Dancer object.
• Figure 3: Overview of our pipeline. We first reconstruct an initial mesh from multi-view images and convert it into a Gaussian point cloud with manifold topology. A topology-aware densification and pruning strategy is then applied to refine the Gaussian representation while preserving surface connectivity. Temporal consistency constraints are introduced to enforce coherent deformation of Gaussians across frames. Finally, topology-consistent Gaussian and mesh sequences are obtained, enabling accurate 3D keypoint tracking.
• Figure 3: Comparison of training results for the Boxer object.