Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuning

TL;DR

Mesh-RFT tackles the challenge of producing high-fidelity, topologically sound 3D meshes by moving beyond global reinforcement signals to fine-grained, face-level refinement. The method combines Masked Direct Preference Optimization (M-DPO) with a topology-aware scoring system that uses Boundary Edge Ratio ($BER$) and Topology Score ($TS$) to guide localized improvements while preserving global coherence. A three-stage pipeline—pretraining, preference dataset construction, and post-training with quality-aware masks—yields substantial gains in geometric integrity and topological regularity, outperforming pretrained baselines and global DPO methods (e.g., $HD$ reductions of up to 24.6% and $TS$ gains of up to 4.9%). The approach establishes a new benchmark for production-ready mesh generation and highlights the value of localized reinforcement learning in 3D asset creation for industry applications.

Abstract

Existing pretrained models for 3D mesh generation often suffer from data biases and produce low-quality results, while global reinforcement learning (RL) methods rely on object-level rewards that struggle to capture local structure details. To address these challenges, we present Mesh-RFT, a novel fine-grained reinforcement fine-tuning framework that employs Masked Direct Preference Optimization (M-DPO) to enable localized refinement via quality-aware face masking. To facilitate efficient quality evaluation, we introduce an objective topology-aware scoring system to evaluate geometric integrity and topological regularity at both object and face levels through two metrics: Boundary Edge Ratio (BER) and Topology Score (TS). By integrating these metrics into a fine-grained RL strategy, Mesh-RFT becomes the first method to optimize mesh quality at the granularity of individual faces, resolving localized errors while preserving global coherence. Experiment results show that our M-DPO approach reduces Hausdorff Distance (HD) by 24.6% and improves Topology Score (TS) by 3.8% over pre-trained models, while outperforming global DPO methods with a 17.4% HD reduction and 4.9% TS gain. These results demonstrate Mesh-RFT's ability to improve geometric integrity and topological regularity, achieving new state-of-the-art performance in production-ready mesh generation. Project Page: https://hitcslj.github.io/mesh-rft/.

Figures (12)

• Figure 1: Representative High-Fidelity Mesh Generation by Mesh-RFT. Gallery of meshes generated from point clouds, demonstrating intricate geometric detail and artist-like aesthetic quality.
• Figure 2: High-quality, artist-like structures often coexist with messy, low-quality regions within the same mesh.
• Figure 3: Mesh-RFT Framework Overview. The pipeline comprises three stages: 1) Mesh Generation Pre-training using an Hourglass AutoRegressive Transformer and a Shape Encoder; 2) Preference Dataset Construction where a pretrained model generates candidate meshes, and a topology-aware score system establishes preference pairs; and 3) Mesh Generation Post-training which employs Mask DPO with reference and policy networks for subsequent refinement.
• Figure 4: Examples of collected preference pairs. Meshes are annotated as preferred using our scoring system. For certain pairs, the selected "good" meshes may exhibit inferior local performance in specific regions compared to the rejected "bad" meshes.
• Figure 5: Qualitative comparison on artist-designed meshes. Our method generates more coherent and visually plausible surfaces with finer structural details and fewer topological artifacts compared to baseline approaches.