Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesis

TL;DR

Trans4D tackles text-to-4D scene synthesis with complex geometry-aware transitions by coupling physics-aware transition planning, powered by multimodal language models, with a geometry-aware Transition Network that governs per-point visibility over time via $p_{trans}$. It leverages 3D Gaussian Splatting to represent dynamic scenes and performs a two-stage training regime: first learning deformation and transition dynamics with SDS-based supervision, then refining 3DGS detail while preserving motion. The approach demonstrates superior performance to state-of-the-art baselines on both quantitative metrics (e.g., QAlign-vid-quality, CLIP-score, MLLM-score) and qualitative assessments, enabling coherent global interactions and significant deformations (e.g., missiles morphing into explosions). This work advances practical, controllable 4D scene generation with potential impact on gaming and video production, and provides code at the project URL for reproducibility.

Abstract

Recent advances in diffusion models have demonstrated exceptional capabilities in image and video generation, further improving the effectiveness of 4D synthesis. Existing 4D generation methods can generate high-quality 4D objects or scenes based on user-friendly conditions, benefiting the gaming and video industries. However, these methods struggle to synthesize significant object deformation of complex 4D transitions and interactions within scenes. To address this challenge, we propose Trans4D, a novel text-to-4D synthesis framework that enables realistic complex scene transitions. Specifically, we first use multi-modal large language models (MLLMs) to produce a physic-aware scene description for 4D scene initialization and effective transition timing planning. Then we propose a geometry-aware 4D transition network to realize a complex scene-level 4D transition based on the plan, which involves expressive geometrical object deformation. Extensive experiments demonstrate that Trans4D consistently outperforms existing state-of-the-art methods in generating 4D scenes with accurate and high-quality transitions, validating its effectiveness. Code: https://github.com/YangLing0818/Trans4D

Figures (8)

• Figure 1: Comparing our Trans4D with Comp4D xu2024comp4d in 4D scene transition generation.
• Figure 2: Overview of our Trans4D, consisting of physics-aware 4D Transition Planning and Transition Network that enable 4D scene generation with complex interaction.
• Figure 3: Qualitative comparison with previous baseline methods bahmani20244dzheng2024unifiedjiangconsistent4dxu2024comp4d. Our method achieves smoother geometric 4D transitions and produces more realistic object interactions within 4D scenes.
• Figure 4: Additional user study for model analysis.
• Figure 5: Ablation study of Physics-aware 4D Transition Planning method.