Generative Plant Growth Simulation from Sequence-Informed Environmental Conditions

TL;DR

This work introduces a sequence-informed plant growth simulation framework (SI-PGS) that employs a conditional generative model to implicitly learn a distribution of possible plant representations within a dynamic scene from a fusion of low-dimensional temporal sensor and context data.

Abstract

A plant growth simulation can be characterized as a reconstructed visual representation of a plant or plant system. The phenotypic characteristics and plant structures are controlled by the scene environment and other contextual attributes. Considering the temporal dependencies and compounding effects of various factors on growth trajectories, we formulate a probabilistic approach to the simulation task by solving a frame synthesis and pattern recognition problem. We introduce a sequence-informed plant growth simulation framework (SI-PGS) that employs a conditional generative model to implicitly learn a distribution of possible plant representations within a dynamic scene from a fusion of low-dimensional temporal sensor and context data. Methods such as controlled latent sampling and recurrent output connections are used to improve coherence in the plant structures between frames of prediction. In this work, we demonstrate that SI-PGS is able to capture temporal dependencies and continuously generate realistic frames of plant growth.

Figures (4)

• Figure 1: Conceptual frame synthesis framework relating the $X$ sequence condition domain to the $Y$ structured output domain.
• Figure 2: Directed graph models. (a) SI-PGS. This model incorporates generation edges that represent the forward generative process and recognition edges that depict the encoding processes of the latent space, $Z$. The conditional vector, $X$, is influenced by the last layer hidden outputs, $h$, of an RNN that processes compounding sequential context data, $c(\tau)$, and is subsequently used to derive structured outputs, $Y$. (b) SI-PGS with reccurent output connections (SI-PGS-R). Recurrent connections are introduced to the model to utilize feedback from the generated output $Y$ to refine $X$ and $Z$.
• Figure 3: SI-PGS network architecture. Input, $c(\tau)$, is composed of a sequence of scene attributes (i.e. temperature, humidity, pH, EC and temporal attributes) that are encoded by the LSTM Encoder $S_\theta$. The encoded output $x$ and the ground truth $Y$ are concatenated and fed into the encoder $F_\phi$, which produces the latent distribution variables $\mu$ and $\sigma^2$. latent variable $z$ is sampled and passed through the decoder $G_\theta$ to generate the output $\hat{y}$. The discriminator $D_\theta$ distinguishes between real and generated outputs. Implementation of the recurrent connection is denoted as SI-PGS-R.
• Figure 4: Comparative generated model outputs. Top two rows show the input signals over time and corresponding real reference frames. Subsequent rows show the outputs of corresponding generative models.