DoubleDiffusion: Combining Heat Diffusion with Denoising Diffusion for Texture Generation on 3D Meshes

TL;DR

This work tackles texture generation for 3D meshes by directly operating on mesh surfaces rather than relying on 2D UV projections. It introduces DoubleDiffusion, which embeds heat-diffusion on the mesh into a denoising diffusion probabilistic model, leveraging the cotangent Laplacian to respect surface geometry. The approach yields geometry-aware, scalable texture synthesis with substantial efficiency gains and improved coverage over the state of the art, demonstrated on single-manifold textures and per-category ShapeNet textures, up to tens of thousands of vertices. The method offers a practical path toward high-fidelity, view-consistent textures for complex 3D assets, enabling texture generation directly in the native mesh domain with strong performance.

Abstract

This paper addresses the problem of generating textures for 3D mesh assets. Existing approaches often rely on image diffusion models to generate multi-view image observations, which are then transformed onto the mesh surface to produce a single texture. However, due to the gap between multi-view images and 3D space, such process is susceptible to arange of issues such as geometric inconsistencies, visibility occlusion, and baking artifacts. To overcome this problem, we propose a novel approach that directly generates texture on 3D meshes. Our approach leverages heat dissipation diffusion, which serves as an efficient operator that propagates features on the geometric surface of a mesh, while remaining insensitive to the specific layout of the wireframe. By integrating this technique into a generative diffusion pipeline, we significantly improve the efficiency of texture generation compared to existing texture generation methods. We term our approach DoubleDiffusion, as it combines heat dissipation diffusion with denoising diffusion to enable native generative learning on 3D mesh surfaces.

Figures (8)

• Figure 1: Overview of the DoubleDiffusion.
• Figure 2: Illustration of the cotangent weighted differential coordinates on the manifold with respect to manifold mesh.
• Figure 3: Method Overview. Our (a) DoubleDiffusion Network contains consecutive (b) DoubleDiffusion Blocks (DDB). Each (c) DDB consists of the Heat Diffusion module aggregating vertice features over the spatial domain on mesh surfaces and the Timestep-Aware MLP that injects timestep embedding in a residual manner.
• Figure 4: Overview of the DoubleDiffusion Block, containing (a) Geometric-aware Low Pass Filter; (b) Spacial gradient Feature and (c) Timestep-aware MLP.
• Figure 5: Qualitative comparison between the (a) MDF (baseline) and (b) DoubleDiffusion (ours) on the Stanford Bunny with $\sim$5k number of vertices. Our method can efficiently handle the mesh with large number of vertices manifold mesh. Here we show our result with (c) $\sim$25k vertices, and (d) $\sim$52k vertices. The meshes with higher vertices are watertight remeshed with the off-the-shelf watertight manifold tool ManifoldPlus huang2020manifoldplus.