Grow with the Flow: 4D Reconstruction of Growing Plants with Gaussian Flow Fields

TL;DR

GrowFlow addresses the challenge of reconstructing time-varying 3D plant growth by modeling growth as a continuous Gaussian flow using a neural ODE over 3D Gaussian splats. By reversing growth from a fully grown plant, the method learns a time-integrated velocity field that continuously evolves Gaussian parameters (centers, rotations, scales) while keeping appearance fixed, enabling new geometry to emerge in a differentiable framework. The approach combines a HexPlane-based encoder with MLP decoders to produce per-parameter velocities, and employs a multi-stage training regime with boundary and global optimization to ensure temporal coherence and geometric fidelity. Experiments on synthetic and real-world timelapse data show state-of-the-art performance in both novel-view and novel-time synthesis, with robust interpolation and strong geometric tracking of plant growth. This framework enables accurate, temporally coherent appearance modeling of growing 3D structures and can be extended to other domains where geometry evolves over time.

Abstract

Modeling the time-varying 3D appearance of plants during their growth poses unique challenges: unlike many dynamic scenes, plants generate new geometry over time as they expand, branch, and differentiate. Recent motion modeling techniques are ill-suited to this problem setting. For example, deformation fields cannot introduce new geometry, and 4D Gaussian splatting constrains motion to a linear trajectory in space and time and cannot track the same set of Gaussians over time. Here, we introduce a 3D Gaussian flow field representation that models plant growth as a time-varying derivative over Gaussian parameters -- position, scale, orientation, color, and opacity -- enabling nonlinear and continuous-time growth dynamics. To initialize a sufficient set of Gaussian primitives, we reconstruct the mature plant and learn a process of reverse growth, effectively simulating the plant's developmental history in reverse. Our approach achieves superior image quality and geometric accuracy compared to prior methods on multi-view timelapse datasets of plant growth, providing a new approach for appearance modeling of growing 3D structures.

Figures (11)

• Figure 1: GrowFlow. We propose GrowFlow, a method for reconstructing high-fidelity geometry of plant growth. Given multi-view timelapse images of a plant, our method accurately reconstructs the dynamic structure using a set of 3D Gaussian primitives and a flow field defined over their parameters. We can also track structures during a plant's growth by visualizing the positions of the 3D Gaussian primitives, as shown above.
• Figure 2: Method overview.a) Our method first optimizes a set of 3D Gaussians on the fully-grown plant. b) Using the optimized 3D Gaussians from the fully-grown plant, we progressively train the dynamics model to learn the state of the plant at each timestep. After each reconstructed timestep, we cache the Gaussians for that timestep and use them as initial conditions to optimize for the next timestep. c) During the global optimization step, we randomly sample a timestep $t_k$ and integrate to $t_{k+1}$, leveraging the cached Gaussians from the boundary reconstruction step as initial conditions. We then optimize the dynamics model to enforce consistency between rendered and captured measurements.
• Figure 3: Results on synthetic data. We compare results on both seen and interpolated times. GrowFlow achieves stable, coherent geometry, unlike prior methods that show visually correct RGB but inconsistent deformations. Yellow marks interpolated frames.
• Figure 4: Results on captured data. We compare results on both seen and interpolated times. GrowFlow achieves stable geometry, unlike prior methods that show visually correct renderings for training frames but struggle on interpolation frames. Yellow marks interpolated frames.
• Figure 5: Qualitative ablations. Replacing our HexPlane representation with an MLP with Fourier encoding reduces capacity and degrades rendering quality. Skipping the boundary reconstruction stage causes the reconstructed geometry to break down.