Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PSGS: Text-driven Panorama Sliding Scene Generation via Gaussian Splatting

Xin Zhang, Shen Chen, Jiale Zhou, Lei Li

TL;DR

PSGS tackles the problem of generating high-fidelity, globally consistent 3D scenes from text by combining a two-layer panorama generation module with a panorama sliding reconstruction pipeline. The first stage uses semantic CoT-guided layout reasoning and iterative MLLM-driven refinement to produce coherent 360° panoramas, while the second stage converts panoramas into overlapping views and initializes a 3D scene with Gaussian Splatting primitives, refined through Gaussian Bundle Adjustment and depth/semantic consistency losses. Key contributions include the two-layer panorama optimization, Panorama Sliding for robust 3D initialization, and joint semantic-geometric losses ($\mathcal{L}_{sem}$ and $\mathcal{L}_{geo}$) that improve cross-view coherence. The approach achieves superior panorama generation and rendering quality compared with state-of-the-art baselines, enabling scalable, immersive content creation with strong global consistency and photorealism, and it lays groundwork for future physics-aware lighting and complex text inputs.

Abstract

Generating realistic 3D scenes from text is crucial for immersive applications like VR, AR, and gaming. While text-driven approaches promise efficiency, existing methods suffer from limited 3D-text data and inconsistent multi-view stitching, resulting in overly simplistic scenes. To address this, we propose PSGS, a two-stage framework for high-fidelity panoramic scene generation. First, a novel two-layer optimization architecture generates semantically coherent panoramas: a layout reasoning layer parses text into structured spatial relationships, while a self-optimization layer refines visual details via iterative MLLM feedback. Second, our panorama sliding mechanism initializes globally consistent 3D Gaussian Splatting point clouds by strategically sampling overlapping perspectives. By incorporating depth and semantic coherence losses during training, we greatly improve the quality and detail fidelity of rendered scenes. Our experiments demonstrate that PSGS outperforms existing methods in panorama generation and produces more appealing 3D scenes, offering a robust solution for scalable immersive content creation.

PSGS: Text-driven Panorama Sliding Scene Generation via Gaussian Splatting

TL;DR

PSGS tackles the problem of generating high-fidelity, globally consistent 3D scenes from text by combining a two-layer panorama generation module with a panorama sliding reconstruction pipeline. The first stage uses semantic CoT-guided layout reasoning and iterative MLLM-driven refinement to produce coherent 360° panoramas, while the second stage converts panoramas into overlapping views and initializes a 3D scene with Gaussian Splatting primitives, refined through Gaussian Bundle Adjustment and depth/semantic consistency losses. Key contributions include the two-layer panorama optimization, Panorama Sliding for robust 3D initialization, and joint semantic-geometric losses ( and ) that improve cross-view coherence. The approach achieves superior panorama generation and rendering quality compared with state-of-the-art baselines, enabling scalable, immersive content creation with strong global consistency and photorealism, and it lays groundwork for future physics-aware lighting and complex text inputs.

Abstract

Generating realistic 3D scenes from text is crucial for immersive applications like VR, AR, and gaming. While text-driven approaches promise efficiency, existing methods suffer from limited 3D-text data and inconsistent multi-view stitching, resulting in overly simplistic scenes. To address this, we propose PSGS, a two-stage framework for high-fidelity panoramic scene generation. First, a novel two-layer optimization architecture generates semantically coherent panoramas: a layout reasoning layer parses text into structured spatial relationships, while a self-optimization layer refines visual details via iterative MLLM feedback. Second, our panorama sliding mechanism initializes globally consistent 3D Gaussian Splatting point clouds by strategically sampling overlapping perspectives. By incorporating depth and semantic coherence losses during training, we greatly improve the quality and detail fidelity of rendered scenes. Our experiments demonstrate that PSGS outperforms existing methods in panorama generation and produces more appealing 3D scenes, offering a robust solution for scalable immersive content creation.
Paper Structure (21 sections, 9 equations, 3 figures, 3 tables)

This paper contains 21 sections, 9 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Two-layer Panorama Generation
  4. Layout Reasoning Layer
  5. Self-optimization Layer
  6. Panorama Sliding Scene Reconstruction
  7. Panorama Sliding
  8. 3D Gaussian Splatting
  9. MASt3R-Based Point Cloud Initialization
  10. Gaussian Bundle Adjustment
  11. Training Objectives
  12. Semantic Similarity Distillation
  13. Geometric Consistency Constraint
  14. Optimization
  15. Experiments
  16. ...and 6 more sections

Figures (3)

  • Figure 1: Method Overview: Two-stage pipeline of the PSGS framework. The first stage propose a novel two-layer architecture that progressively optimizes scene generation through semantic reasoning and visual feedback. The second stage establishes globally consistent 3D reconstruction through panorama sliding, combined with semantic and geometric consistency constraints for high-fidelity rendering.
  • Figure 2: Sample panoramas generated by LucidDreamer, DreamScene360 and our method under identical prompts.
  • Figure 3: Qualitative comparison of rendering results between our method and baseline approaches. Our method produces images with higher fidelity, better geometric details across different viewpoints.