WorldGrow: Generating Infinite 3D World

TL;DR

WorldGrow addresses the challenge of infinite, coherent 3D world generation by proposing a block-based growth framework trained on curated scene blocks. It integrates scene-friendly SLAT representations, 3D block inpainting, and a coarse-to-fine generation pipeline to ensure global layout plausibility and local detail in unbounded environments. On 3D-FRONT, it achieves state-of-the-art geometry reconstruction and renders photorealistic blocks, while enabling scalable, continuous expansion with minimal seams. This work advances toward practical large-scale virtual environments and future world models for embodied AI.

Abstract

We tackle the challenge of generating the infinitely extendable 3D world -- large, continuous environments with coherent geometry and realistic appearance. Existing methods face key challenges: 2D-lifting approaches suffer from geometric and appearance inconsistencies across views, 3D implicit representations are hard to scale up, and current 3D foundation models are mostly object-centric, limiting their applicability to scene-level generation. Our key insight is leveraging strong generation priors from pre-trained 3D models for structured scene block generation. To this end, we propose WorldGrow, a hierarchical framework for unbounded 3D scene synthesis. Our method features three core components: (1) a data curation pipeline that extracts high-quality scene blocks for training, making the 3D structured latent representations suitable for scene generation; (2) a 3D block inpainting mechanism that enables context-aware scene extension; and (3) a coarse-to-fine generation strategy that ensures both global layout plausibility and local geometric/textural fidelity. Evaluated on the large-scale 3D-FRONT dataset, WorldGrow achieves SOTA performance in geometry reconstruction, while uniquely supporting infinite scene generation with photorealistic and structurally consistent outputs. These results highlight its capability for constructing large-scale virtual environments and potential for building future world models.

Figures (9)

• Figure 1: We introduce WorldGrow, a novel framework for infinite 3D world generation via block-wise synthesis and growth with coarse-to-fine refinement. Starting from a single seed block, WorldGrow progressively generates large-scale 3D scenes with coherent geometry and photorealistic appearance. Top: An indoor scene of $19\times39$ blocks (covering $\sim$1,800 m$^2$); left→right: coarse voxel layout, refined voxels, mesh reconstruction, and textured rendering. Bottom: An embodied agent navigates the generated world, demonstrating diverse room layouts and traversable spaces suitable for interactive AI tasks (e.g., navigation and planning).
• Figure 2: Overview of WorldGrow. Our goal is to generate infinite 3D scenes through modular, block-by-block synthesis. We begin by curating high-quality scene blocks and adapting SLAT to better model structured 3D context. A 3D block inpainting module enables spatially coherent extension, while a coarse-to-fine generation strategy ensures global layout plausibility and local detail fidelity. Together, these components allow WorldGrow to progressively construct photorealistic and structurally consistent 3D worlds with theoretically unbounded spatial extent.
• Figure 3: Scene-friendly SLAT better models 3D scene blocks, particularly in areas with occlusions and near block boundaries.
• Figure 4: 1D illustration of our block-by-block expansion. Existing block's $[1/2w,\; 7/8w)$ area is used as context for inpainting the next block. Thus, the final region $[7/8w,\; w)$ is discarded and then re-generated during expansion.
• Figure 5: Qualitative comparison of indoor scene generation. We compare our method with state-of-the-art infinite scene generation approaches, indoor house generation methods and our baseline TRELLIS. WorldGrow produces high-resolution, continuous indoor scenes with realistic and coherent textures.