Apply Hierarchical-Chain-of-Generation to Complex Attributes Text-to-3D Generation

TL;DR

This work tackles the challenge of generating 3D objects with complex attributes from long text prompts. It introduces Hierarchical-Chain-of-Generation (HCoG), which uses a large language model to decompose descriptions into occlusion-ordered hierarchical blocks, and performs coarse-to-fine, part-specific optimization on 3D Gaussian Splatting with automated Gaussian Extension and Label Elimination to add new parts without disturbing existing ones. Within each block, fine-grained attribute binding is achieved via part segmentation, ControlNet-based shape priors, and SDS-based optimization, enabling accurate attribute localization and multi-view consistency. Empirical results, including qualitative comparisons and quantitative metrics like BLIP-VQA and CLIP scores, show that HCoG achieves structurally coherent, attribute-faithful 3D assets and scales across backbones, significantly reducing manual guidance requirements. The approach promises automated, scalable generation of complex 3D assets suitable for adoption in 3D content creation pipelines and related AI-assisted design tools.

Abstract

Recent text-to-3D models can render high-quality assets, yet they still stumble on objects with complex attributes. The key obstacles are: (1) existing text-to-3D approaches typically lift text-to-image models to extract semantics via text encoders, while the text encoder exhibits limited comprehension ability for long descriptions, leading to deviated cross-attention focus, subsequently wrong attribute binding in generated results. (2) Occluded object parts demand a disciplined generation order and explicit part disentanglement. Though some works introduce manual efforts to alleviate the above issues, their quality is unstable and highly reliant on manual information. To tackle above problems, we propose a automated method Hierarchical-Chain-of-Generation (HCoG). It leverages a large language model to decompose the long description into blocks representing different object parts, and orders them from inside out according to occlusions, forming a hierarchical chain. Within each block we first coarsely create components, then precisely bind attributes via target-region localization and corresponding 3D Gaussian kernel optimization. Between blocks, we introduce Gaussian Extension and Label Elimination to seamlessly generate new parts by extending new Gaussian kernels, re-assigning semantic labels, and eliminating unnecessary kernels, ensuring that only relevant parts are added without disrupting previously optimized parts. Experiments confirm that HCoG yields structurally coherent, attribute-faithful 3D objects with complex attributes. The code is available at https://github.com/Wakals/GASCOL .

Figures (9)

• Figure 1: The problem of existing work and the example of our method.
• Figure 2: Overview of Hierarchical-Chain-of-Generation. a) In the Hierarchical Blocks stage, LLM analyzes the input text and based on the order from more occlusion to less occlusion, creating the order of generation. b) Part-optimization is applied to the parts in blocks, using Lang-SAM luca2024langSAM to segment specific parts and utilizing MVDream Shi2023MVDreamMD and ControlNet Zhang2023AddingCCControlNet in fine-grained optimization stage to enable corresponding attributes binding for each part with shape and multi-view consistency. c) Gaussian Extension is applied between blocks, extending new parts for the next block. d) Label Elimination aims to generate new parts by extending new Gaussian kernels (red-star-marked), re-assigning semantic labels (blue-star-marked), and eliminating unnecessary kernels finally, ensuring that only relevant parts are generated without disrupting previously optimized parts.
• Figure 3: Visual comparison with other methods. We compare our method with other well performed text-to-3D methods Yi2023GaussianDreamerFGLiang2023LucidDreamerTHShi2023MVDreamMD, Progressive3D Cheng2023Progressive3DPL which heavily relies on user-defined generation order and bounding boxes, and Stable Diffusion v3 Rombach2021HighResolutionISSD which is a more powerful backend.
• Figure 4: Ablation study of ControlNet. Input text: blue sports shoes. Without shape control, the diffusion model will give wrong guidance and the result will be bad.
• Figure 5: Ablation study of Label Elimination. Previous optimized input text: A man in yellow shirt, pink trousers and blue leather shoes is waving. Next input text: A man in coat is waving. Without Label Elimination, when generating new part coat, the optimized parts like yellow shirt, pink trousers and blue leather shoes are changed.