Topology Sculptor, Shape Refiner: Discrete Diffusion Model for High-Fidelity 3D Meshes Generation

TL;DR

This work introduces Topology Sculptor, Shape Refiner (TSSR), a token-parallel Diffusion Model framework for direct generation of high-fidelity 3D meshes. By decoupling training into topology sculpting and shape refinement, employing an enhanced Hourglass transformer with multi-level RoPE, and enforcing topology through a novel connection loss, TSSR preserves accurate token prediction despite parallel generation. Empirical results on ShapeNet, Thingi10K, and Objaverse show state-of-the-art performance for artist-style meshes, generating up to 10K faces at a $1024^3$ resolution with efficient inference. The approach offers a practical pathway to high-quality, parallel mesh synthesis with strong topology control and broad applicability in digital content creation.

Abstract

In this paper, we introduce Topology Sculptor, Shape Refiner (TSSR), a novel method for generating high-quality, artist-style 3D meshes based on Discrete Diffusion Models (DDMs). Our primary motivation for TSSR is to achieve highly accurate token prediction while enabling parallel generation, a significant advantage over sequential autoregressive methods. By allowing TSSR to "see" all mesh tokens concurrently, we unlock a new level of efficiency and control. We leverage this parallel generation capability through three key innovations: 1) Decoupled Training and Hybrid Inference, which distinctly separates the DDM-based generation into a topology sculpting stage and a subsequent shape refinement stage. This strategic decoupling enables TSSR to effectively capture both intricate local topology and overarching global shape. 2) An Improved Hourglass Architecture, featuring bidirectional attention enriched by face-vertex-sequence level Rotational Positional Embeddings (RoPE), thereby capturing richer contextual information across the mesh structure. 3) A novel Connection Loss, which acts as a topological constraint to further enhance the realism and fidelity of the generated meshes. Extensive experiments on complex datasets demonstrate that TSSR generates high-quality 3D artist-style meshes, capable of achieving up to 10,000 faces at a remarkable spatial resolution of $1024^3$. The code will be released at: https://github.com/psky1111/Tencent-TSSR.

Figures (4)

• Figure 1: An illustration of different types of noise for meshes. Then [MASK] is treated as a token removal. As a result, mask-based noise will force the model to focus on local structural completion. Uniform noise will generate a "hedgehog"-like mesh, leading the model to focus on the shape.
• Figure 2: An illustration of improved Hourglass architecture. (a) illustrates the overall framework, where we add $c_{flag}$, an additional light-aware classifier to represent $\phi$. The classifier heads serve as initial heads for predicting tokens. (b) illustrates that we change the initial RoPE to a multi-level RoPE, and (c) illustrates the multi-level RoPE.
• Figure 3: Qualitative results about the generated mesh. It can be found that the shape of the AR-based method will lose some parts. TSSR can benefit greatly from token parallel generation. Meanwhile, the topology of TSSR maintains a similar quality to that of AR's methods.
• Figure 4: Qualitative results about the ablation study of connection loss.