SuperCarver: Texture-Consistent 3D Geometry Super-Resolution for High-Fidelity Surface Detail Generation

TL;DR

This work tackles the challenge of upgrading geometry fidelity for existing 3D meshes while preserving texture coherence. It introduces SuperCarver, a two-stage pipeline that first ups the detail of texture-aligned normals via deterministic prior-guided diffusion in image space, and then transfers these details back to the 3D surface through a noise-resistant inverse rendering driven by a deformable distance-field. Key contributions include a mesh-to-mesh geometry super-resolution framework, a deterministic diffusion process conditioned on rich priors, and a differentiable, distance-field-based mesh refinement strategy that yields texture-consistent, high-fidelity geometry. Experiments demonstrate improved normal predictions and mesh quality across diverse assets, with runtimes of about 1–2 minutes, indicating practical viability for upgrading legacy assets and reducing manual sculpting workload. The approach offers a promising direction for texture-guided 3D detail generation and invites further exploration of faster, feed-forward alternatives and more flexible implicit geometry representations.

Abstract

Conventional production workflow of high-precision mesh assets necessitates a cumbersome and laborious process of manual sculpting by specialized 3D artists/modelers. The recent years have witnessed remarkable advances in AI-empowered 3D content creation for generating plausible structures and intricate appearances from images or text prompts. However, synthesizing realistic surface details still poses great challenges, and enhancing the geometry fidelity of existing lower-quality 3D meshes (instead of image/text-to-3D generation) remains an open problem. In this paper, we introduce SuperCarver, a 3D geometry super-resolution pipeline for supplementing texture-consistent surface details onto a given coarse mesh. We start by rendering the original textured mesh into the image domain from multiple viewpoints. To achieve detail boosting, we construct a deterministic prior-guided normal diffusion model, which is fine-tuned on a carefully curated dataset of paired detail-lacking and detail-rich normal map renderings. To update mesh surfaces from potentially imperfect normal map predictions, we design a noise-resistant inverse rendering scheme through deformable distance field. Experiments demonstrate that our SuperCarver is capable of generating realistic and expressive surface details depicted by the actual texture appearance, making it a powerful tool to both upgrade historical low-quality 3D assets and reduce the workload of sculpting high-poly meshes.

Figures (18)

• Figure 1: Given an input 3D textured model lacking geometric details, SuperCarver generates realistic and expressive 3D surface details consistent with its texture appearance within 1 $\sim$ 2 minutes, significantly enhancing the production efficiency of high-quality mesh assets. (Row 1: The input 3D textured meshes. Rows 2&4: The original coarse surfaces and normal maps. Rows 3&5: The resulting high-fidelity mesh surfaces and normal maps after our geometry super-resolution.)
• Figure 2: The overall processing pipeline of our proposed SuperCarver for texture-consistent geometry super-resolution. Given an input 3D textured model, we begin by rendering color, depth, and normal maps, which are fed into a customized deterministic prior-guided diffusion architecture to generate detail-boosted normal maps. To perform gradient-based mesh refinement from potentially imperfect multi-view normal map predictions, we further design a noise-resistant distance field deformation process to drive the optimization of high-fidelity 3D surface details.
• Figure 3: Results of applying our proposed SuperCarver for texture-consistent 3D geometry super-resolution.
• Figure 4: Illustration of applying our proposed SuperCarver for further 3D surface detail generation on textured meshes generated from (a) InstantMesh xu2024instantmesh, (b) Unique3D wu2024unique3d, and (c) CLAY zhang2024clay.
• Figure 5: Results of applying our proposed SuperCarver on textured CAD mesh models collected from ShapeNet chang2015shapenet.