InsTex: Indoor Scenes Stylized Texture Synthesis

TL;DR

InsTex tackles the challenge of generating high‑quality, style‑consistent textures for indoor 3D scenes across diverse objects and viewpoints. It introduces a two‑stage pipeline that uses depth‑to‑image diffusion priors, scene decomposition, and UV‑space refinement with global style guidance to produce textures conditioned on textual or visual prompts. The approach includes canonical space objectization, coarse multi‑view texture generation with dynamic view partitioning, adjacency‑conditioned UV refinement, and a final scene re‑composition with post‑processing, achieving state‑of‑the‑art texture quality and fidelity on 3D‑FRONT datasets. The methodology offers practical impact for AR/VR interior design and gaming by delivering fast, consistent, and configurable texture synthesis for complex indoor environments.

Abstract

Generating high-quality textures for 3D scenes is crucial for applications in interior design, gaming, and augmented/virtual reality (AR/VR). Although recent advancements in 3D generative models have enhanced content creation, significant challenges remain in achieving broad generalization and maintaining style consistency across multiple viewpoints. Current methods, such as 2D diffusion models adapted for 3D texturing, suffer from lengthy processing times and visual artifacts, while approaches driven by 3D data often fail to generalize effectively. To overcome these challenges, we introduce InsTex, a two-stage architecture designed to generate high-quality, style-consistent textures for 3D indoor scenes. InsTex utilizes depth-to-image diffusion priors in a coarse-to-fine pipeline, first generating multi-view images with a pre-trained 2D diffusion model and subsequently refining the textures for consistency. Our method supports both textual and visual prompts, achieving state-of-the-art results in visual quality and quantitative metrics, and demonstrates its effectiveness across various 3D texturing applications.

Figures (5)

• Figure 1: InsTex Pipeline: The proposed InsTex pipeline starts with an untextured scene mesh and a style prompt, such as "baroque bedroom" in this case. It then generates a high-quality, visually pleasing, and stylized "baroque" living room, with textures that are conditioned on the given prompt.
• Figure 2: Qualitative comparisons. Latent-Paint metzer2022latent suffers from issues of over-saturation and the hallucination of scene elements, while MVDiffusion tang2023mvdiffusion produces blurry textures that do not accurately reflect the input prompts. Text2Tex chen2023text2tex encounters difficulties in maintaining style consistency across all instances. SceneTex chen2023scenetexhighqualitytexturesynthesis shows some artifacts in both floor and wall. Our method generates high-quality textures while preserving overall style consistency throughout the scenes.
• Figure 3: Synthesized textures for 3D-FRONT scenes. Our method generates high-quality style-coherent textures, and reflects the iconic traits in the prompts.
• Figure 4: The comparison of the textured table. The first raw shows the coarse stage outputs, the second row shows the the refinement stage outputs. After refinement, the texture quality is improved with better visual quality and the missing parts like the missing joints are in-painted as well.
• Figure 5: The texturization process for the Baroque coffee table was evaluated under varying diffusion step settings during the coarse stage. When utilizing 50 diffusion steps, the generated textures exhibited vivid details and achieved optimal alignment with the distinctive Baroque style, demonstrating the effectiveness of this configuration in capturing intricate stylistic elements.