FastMesh: Efficient Artistic Mesh Generation via Component Decoupling

TL;DR

FastMesh tackles inefficiency in autoregressive mesh generation by decoupling vertex and face construction. It autoregressively generates vertices with block-wise indexing, then uses a bidirectional transformer to infer edges and assemble faces in a single step, supplemented by a fidelity enhancer and a prediction-filtering post-process. The approach achieves approximately 23% token usage and up to an 8x speedup on Toys4K while delivering higher mesh quality than prior methods. This decoupled pipeline enables faster, more robust artistic mesh generation conditioned on shape inputs and is compatible with broader 3D generation pipelines.

Abstract

Recent mesh generation approaches typically tokenize triangle meshes into sequences of tokens and train autoregressive models to generate these tokens sequentially. Despite substantial progress, such token sequences inevitably reuse vertices multiple times to fully represent manifold meshes, as each vertex is shared by multiple faces. This redundancy leads to excessively long token sequences and inefficient generation processes. In this paper, we propose an efficient framework that generates artistic meshes by treating vertices and faces separately, significantly reducing redundancy. We employ an autoregressive model solely for vertex generation, decreasing the token count to approximately 23% of that required by the most compact existing tokenizer. Next, we leverage a bidirectional transformer to complete the mesh in a single step by capturing inter-vertex relationships and constructing the adjacency matrix that defines the mesh faces. To further improve the generation quality, we introduce a fidelity enhancer to refine vertex positioning into more natural arrangements and propose a post-processing framework to remove undesirable edge connections. Experimental results show that our method achieves more than 8x faster speed on mesh generation compared to state-of-the-art approaches, while producing higher mesh quality.

Figures (8)

• Figure 1: Example of meshes generated by FastMesh. Our approach efficiently produces 3D objects by substantially reducing the number of tokens required for generation. Note that all meshes are directly generated from point clouds.
• Figure 2: (a) Overall architecture of FastMesh. Note that our pipeline consists of two stages, where we first generate the vertices from the shape condition and then construct the faces to complete the mesh. (b) Visualization of the block-wise indexing scheme introduced by BPT weng2024scaling, which we adopt for vertex tokenization. (c) Structure of the fidelity enhancer in the first stage. The 7-bit discretized vertices and shape condition are fed into the network to estimate the offset that can make the coordinate a continuous value. (d) Details of face reconstruction. The generated vertices are embedded to capture inter-vertex relationships in a multi-head manner. Each head computes a matrix, where the output represents one feature dimension used in edge prediction.
• Figure 3: (a) Example of high-resolution mesh containing 7,694 vertices. (b) Mesh obtained by discretizing (a) in 7-bit coordinate space, resulting in 3,636 vertices. (c) Mesh reconstructed using the same 3,636 vertices as in (b), with continuous coordinates refined by our fidelity enhancer.
• Figure 4: The detailed structure of the prediction filtering. We use the initial adjacency matrix from the first face generation to perform BFS reordering. Based on this reordering, we apply the minimum bandwidth mask and the minimum candidate mask as attention masks.
• Figure 5: Qualitative comparison of shape-conditioned mesh generation on the Toys4K dataset toys4K. All meshes were generated from the same input point clouds that were sampled from the original meshes.