LumiTex: Towards High-Fidelity PBR Texture Generation with Illumination Context

TL;DR

LumiTex tackles the challenge of generating physically plausible PBR textures with illumination-aware material decomposition and seamless multi-view completion. It introduces a three-part pipeline: a multi-branch generation scheme with a multi-view illumination context branch, a lighting-aware material attention mechanism that decouples albedo and MR using shared illumination priors, and a geometry-guided LVSM for dense, view-consistent texture inpainting. The approach leverages a Multi-View PBR Generation Transformer with a Multi-Modal DiT and a cross-view MV-T, guided by shaded priors to improve physical plausibility and cross-view consistency, and a LVSM-based novel-view synthesis module to densify texture coverage. Empirical results show LumiTex outperforms both open-source and commercial baselines in texture quality, relighting fidelity, and perceptual alignment, illustrating its potential to streamline high-quality PBR texture generation for diverse 3D assets.

Abstract

Physically-based rendering (PBR) provides a principled standard for realistic material-lighting interactions in computer graphics. Despite recent advances in generating PBR textures, existing methods fail to address two fundamental challenges: 1) materials decomposition from image prompts under limited illumination cues, and 2) seamless and view-consistent texture completion. To this end, we propose LumiTex, an end-to-end framework that comprises three key components: (1) a multi-branch generation scheme that disentangles albedo and metallic-roughness under shared illumination priors for robust material understanding, (2) a lighting-aware material attention mechanism that injects illumination context into the decoding process for physically grounded generation of albedo, metallic, and roughness maps, and (3) a geometry-guided inpainting module based on a large view synthesis model that enriches texture coverage and ensures seamless, view-consistent UV completion. Extensive experiments demonstrate that LumiTex achieves state-of-the-art performance in texture quality, surpassing both existing open-source and commercial methods.

Figures (9)

• Figure 1: A collection of textured meshes with PBR materials generated by LumiTex, capturing both intricate detail and convincing physical realism across diverse object categories under environmental lighting. Our demo is available at: https://lumitex.vercel.app.
• Figure 2: Illustration of Different PBR Modeling. Unlike (a) two-stage PBR texture generation with suboptimal intermediates, and (b) end-to-end approach w/o multi-view shaded features, (c) our multi-branch design leverages multi-view consistent lighting features and surpasses the generation quality of other models.
• Figure 3: Overview of LumiTex. We first train a multi-view illumination-consistent base model to generate shaded images. Then, we freeze this branch and utilize its shaded features to train material branches for high-fidelity PBR texture generation. Finally, our geometry-guided LVSM synthesizes novel views from novel perspectives to enable seamless, view-consistent texture inpainting.
• Figure 4: Texture Inpainting with Geometry-guided LVSM. Our model infers dense novel views from sparse inputs for inpainting.
• Figure 5: Qualitative Results on Texture Generation Methods. Our method generates plausible materials for relighting, avoids baked-in lighting, and is robust under diverse reference illuminations, including challenging cases like strong backlighting (last row).