CraftMesh: High-Fidelity Generative Mesh Manipulation via Poisson Seamless Fusion

TL;DR

CraftMesh tackles the problem of controllable, high-fidelity mesh editing by decoupling editing into a 2D image edit, 3D region mesh generation, and seamless fusion. It introduces a Joint Geometry and Appearance Fusion framework built on a hybrid SDF/Mesh representation, with Poisson Geometry Blending for geometric continuity and Poisson Texture Harmonization for coherent appearance. The key contributions include the first image-editing–mesh-generation–seamless-fusion pipeline, two core Poisson-based fusion components, and demonstrated compatibility with drag-based editing. Experimental results show superior global consistency, local detail, and color harmony over state-of-the-art baselines, indicating strong practical impact for 3D content creation pipelines.

Abstract

Controllable, high-fidelity mesh editing remains a significant challenge in 3D content creation. Existing generative methods often struggle with complex geometries and fail to produce detailed results. We propose CraftMesh, a novel framework for high-fidelity generative mesh manipulation via Poisson Seamless Fusion. Our key insight is to decompose mesh editing into a pipeline that leverages the strengths of 2D and 3D generative models: we edit a 2D reference image, then generate a region-specific 3D mesh, and seamlessly fuse it into the original model. We introduce two core techniques: Poisson Geometric Fusion, which utilizes a hybrid SDF/Mesh representation with normal blending to achieve harmonious geometric integration, and Poisson Texture Harmonization for visually consistent texture blending. Experimental results demonstrate that CraftMesh outperforms state-of-the-art methods, delivering superior global consistency and local detail in complex editing tasks.

Figures (6)

• Figure 1: Mesh editing results produced by CraftMesh. CraftMesh is a versatile 3D mesh editing framework that enables users to perform text-based and drag-based editing, and delivers high quality outputs even for challenging editing scenarios.
• Figure 2: An overview of the CraftMesh framework. Our framework follows an image editing–mesh generation–seamless fusion pipeline that fully leverages the strengths of 2D models for image editing and 3D models for high-quality mesh generation. First, Edited Region Mesh Generation produces meshes for the editing region. Then, Poisson Geometry Blending achieves natural geometric transitions through normal blending. Finally, Poisson Texture Harmonization performs texture fusion to seamlessly color the edited regions.
• Figure 3: Details of Poisson Geometry Blending. We employ a hybrid SDF/Mesh representation, which is optimized to be harmonious using blended normals.
• Figure 4: Qualitative comparisons show that our method produces harmonious geometry structure, intricate local details, and high-fidelity colors, while other methods give simple and inconsistent results.
• Figure 5: Our method effectively adapts to drag-based mesh editing, enabling precise and intuitive geometry manipulation through user-specified drag controls. (a) shows the original mesh, with arrows drawn to signify drag; (b) shows the editing results.