DynamicTree: Interactive Real Tree Animation via Sparse Voxel Spectrum

TL;DR

DynamicTree tackles realistic, long-term 4D animation of real-world trees reconstructed with 3D Gaussian Splatting by introducing a sparse voxel spectrum to represent mesh motion and a two-stage mesh-to-3DGS deformation pipeline. Motion is encoded in the temporal frequency domain via FFT on sparse voxels, enabling compact, controllable generation that is then devoxelized and bound to Gaussian primitives for full 3DGS animation. The framework further supports real-time interaction through modal analysis, treating the mesh as a mass-spring-damper system and solving in modal space to achieve ~18 ms simulation per frame. To train and evaluate, the authors build 4DTree, a large synthetic dataset with 8,786 animated trees and 100-frame sequences, and demonstrate superior visual quality, temporal coherence, and efficiency over state-of-the-art 4D and physics-based baselines on real and synthetic trees.

Abstract

Generating dynamic and interactive 3D trees has wide applications in virtual reality, games, and world simulation. However, existing methods still face various challenges in generating structurally consistent and realistic 4D motion for complex real trees. In this paper, we propose DynamicTree, the first framework that can generate long-term, interactive 3D motion for 3DGS reconstructions of real trees. Unlike prior optimization-based methods, our approach generates dynamics in a fast feed-forward manner. The key success of our approach is the use of a compact sparse voxel spectrum to represent the tree movement. Given a 3D tree from Gaussian Splatting reconstruction, our pipeline first generates mesh motion using the sparse voxel spectrum and then binds Gaussians to deform the mesh. Additionally, the proposed sparse voxel spectrum can also serve as a basis for fast modal analysis under external forces, allowing real-time interactive responses. To train our model, we also introduce 4DTree, the first large-scale synthetic 4D tree dataset containing 8,786 animated tree meshes with 100-frame motion sequences. Extensive experiments demonstrate that our method achieves realistic and responsive tree animations, significantly outperforming existing approaches in both visual quality and computational efficiency.

Figures (8)

• Figure 1: DynamicTree achieves structurally consistent and realistic long-term animation and dynamic interaction on real-world 3DGS trees. It first generates mesh motion via a compact sparse voxel spectrum representation and then deforms the surface-bound Gaussian primitives. We visualize the slice of the generated motion at the orange scanline along the time dimension.
• Figure 2: Our framework animates 3DGS trees in two stages: (1) spectrum-based motion generation in the frequency domain, and (2) deformation transfer to 3DGS through mesh binding. In the first stage, we extract the tree mesh from multi-view images, voxelize it, and encode it into a sparse voxel latent condition. A sparse voxel diffusion model then generates a compact motion representation $\mathcal{S}$, which is used to reconstruct mesh motion via devoxelization and inverse Fast Fourier Transform. In the second stage, 3DGS primitives are bound to the mesh surface and animated by its deformations.
• Figure 3: Comparison with 4D generation methods. We visualize the middle frame of the sequence, where our method better preserves 3D structures. Space-time slices are shown, with vertical and horizontal axes representing time and the spatial profile along the brown line.
• Figure 4: Interactive simulation comparison of different methods. We apply a dragging external force and then visualize the response of the scene, where our approach produces more natural oscillatory motion with finer-grained details. $t$ and $T$ denote the middle and final frames, respectively.
• Figure 5: Ablation of training strategies. Columns 2–4 show the middle frame of sequences generated by each strategy.