EucliDreamer: Fast and High-Quality Texturing for 3D Models with Depth-Conditioned Stable Diffusion

TL;DR

EucliDreamer tackles automatic 3D texture generation from text prompts by representing textures as an implicit surface function and optimizing via Score Distillation Sampling guided by a depth-conditioned diffusion model. A hash-grid texture representation and differentiable rendering enable efficient optimization, with depth information from rendered meshes reducing ambiguity and improving cross-view consistency. On Objaverse models and through a professional user study, EucliDreamer demonstrates superior texture quality and faster convergence than prior methods, while supporting varied art styles through text prompts. The approach offers a practical, scalable solution to automate labor-intensive 3D asset creation, albeit with limitations on occluded surfaces and baked-in lighting artifacts that invite future work on handling holes and flat lighting variants.

Abstract

We present EucliDreamer, a simple and effective method to generate textures for 3D models given text prompts and meshes. The texture is parametrized as an implicit function on the 3D surface, which is optimized with the Score Distillation Sampling (SDS) process and differentiable rendering. To generate high-quality textures, we leverage a depth-conditioned Stable Diffusion model guided by the depth image rendered from the mesh. We test our approach on 3D models in Objaverse and conducted a user study, which shows its superior quality compared to existing texturing methods like Text2Tex. In addition, our method converges 2 times faster than DreamFusion. Through text prompting, textures of diverse art styles can be produced. We hope Euclidreamer proides a viable solution to automate a labor-intensive stage in 3D content creation.

Figures (15)

• Figure 1: 3D objects textured by EucliDreamer. The generated textures are realistic and highly detailed.
• Figure 2: Our method. Multiple views of the texured mesh are produced by a differentiable renderer, then the SDS loss will back-propagate to update a hash grid, conditioned on text prompts and depth images.
• Figure 3: Comparisons with existing texturing methods. Four objects are used for illustration from Objaverse deitke2022objaverse. The rendering performance of the first three methods, CLIPMesh, Latent-Paint, and Text2Tex are discussed in chen2023text2tex. Overall, the examples demonstrate a clear win of EucliDreamer over the baselines Mohammad_Khalid_2022metzer2022latentnerfchen2023text2tex in terms of clarity, sharpness and level of details.
• Figure 4: Vote distribution for best quality. Ours was selected as the best for most of the objects.
• Figure 6: The SDS loss throughout the iterations. It takes Euclidreamer around 4300 steps, and even 2500 steps gives decent result already.