Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synthesizing Forestry Images Conditioned on Plant Phenotype Using a Generative Adversarial Network

Debasmita Pal, Arun Ross

TL;DR

This work tackles the challenge of visualizing forest sites with controllable plant phenotypes by generating synthetic forestry imagery conditioned on a continuous GCC attribute over a ROI describing a vegetation type. It introduces a CGAN framework with ROI masking and continuous conditioning, augmented by self-attention and spectral normalization, and evaluates the realism and phenotypic fidelity of generated images using SSIM and FID, while measuring GCC and predicting RCC via RMSPE. The study demonstrates site-specific modeling on NEON PhenoCam data (Harvard and Bartlett), cross-site transfer, and scalability to other vegetation types, showing the model can produce diverse, phenotype-consistent images and infer additional attributes from synthetic samples. The proposed approach offers a pathway for visualizing phenology-driven forestry scenarios and could inform ecological analysis and remote sensing interpretation, with future work aimed at diffusion-based generation and incorporating broader environmental parameters.

Abstract

Plant phenology and phenotype prediction using remote sensing data are increasingly gaining attention within the plant science community as a promising approach to enhance agricultural productivity. This work focuses on generating synthetic forestry images that satisfy certain phenotypic attributes, viz. canopy greenness. We harness a Generative Adversarial Network (GAN) to synthesize biologically plausible and phenotypically stable forestry images conditioned on the greenness of vegetation (a continuous attribute) over a specific region of interest, describing a particular vegetation type in a mixed forest. The training data is based on the automated digital camera imagery provided by the National Ecological Observatory Network (NEON) and processed by the PhenoCam Network. Our method helps render the appearance of forest sites specific to a greenness value. The synthetic images are subsequently utilized to predict another phenotypic attribute, viz., redness of plants. The quality of the synthetic images is assessed using the Structural SIMilarity (SSIM) index and Fréchet Inception Distance (FID). Further, the greenness and redness indices of the synthetic images are compared against those of the original images using Root Mean Squared Percentage Error (RMSPE) to evaluate their accuracy and integrity. The generalizability and scalability of our proposed GAN model are established by effectively transforming it to generate synthetic images for other forest sites and vegetation types. From a broader perspective, this approach could be leveraged to visualize forestry based on different phenotypic attributes in the context of various environmental parameters.

Synthesizing Forestry Images Conditioned on Plant Phenotype Using a Generative Adversarial Network

TL;DR

This work tackles the challenge of visualizing forest sites with controllable plant phenotypes by generating synthetic forestry imagery conditioned on a continuous GCC attribute over a ROI describing a vegetation type. It introduces a CGAN framework with ROI masking and continuous conditioning, augmented by self-attention and spectral normalization, and evaluates the realism and phenotypic fidelity of generated images using SSIM and FID, while measuring GCC and predicting RCC via RMSPE. The study demonstrates site-specific modeling on NEON PhenoCam data (Harvard and Bartlett), cross-site transfer, and scalability to other vegetation types, showing the model can produce diverse, phenotype-consistent images and infer additional attributes from synthetic samples. The proposed approach offers a pathway for visualizing phenology-driven forestry scenarios and could inform ecological analysis and remote sensing interpretation, with future work aimed at diffusion-based generation and incorporating broader environmental parameters.

Abstract

Plant phenology and phenotype prediction using remote sensing data are increasingly gaining attention within the plant science community as a promising approach to enhance agricultural productivity. This work focuses on generating synthetic forestry images that satisfy certain phenotypic attributes, viz. canopy greenness. We harness a Generative Adversarial Network (GAN) to synthesize biologically plausible and phenotypically stable forestry images conditioned on the greenness of vegetation (a continuous attribute) over a specific region of interest, describing a particular vegetation type in a mixed forest. The training data is based on the automated digital camera imagery provided by the National Ecological Observatory Network (NEON) and processed by the PhenoCam Network. Our method helps render the appearance of forest sites specific to a greenness value. The synthetic images are subsequently utilized to predict another phenotypic attribute, viz., redness of plants. The quality of the synthetic images is assessed using the Structural SIMilarity (SSIM) index and Fréchet Inception Distance (FID). Further, the greenness and redness indices of the synthetic images are compared against those of the original images using Root Mean Squared Percentage Error (RMSPE) to evaluate their accuracy and integrity. The generalizability and scalability of our proposed GAN model are established by effectively transforming it to generate synthetic images for other forest sites and vegetation types. From a broader perspective, this approach could be leveraged to visualize forestry based on different phenotypic attributes in the context of various environmental parameters.
Paper Structure (20 sections, 5 equations, 21 figures)

This paper contains 20 sections, 5 equations, 21 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Objective and Contribution
  4. Related Work
  5. Generative Adversarial Network (GAN)
  6. Applications in Agriculture
  7. Methodologies
  8. Experiments
  9. Training Individual Sites with a Specific Vegetation Type
  10. Assessing Quality of Synthetic Images
  11. Fidelity and Diversity of Synthetic Images
  12. Evaluating Accuracy of Computed GCC and Predicted RCC
  13. Evaluating Efficacy of our GAN model
  14. Analyzing Significance of Proposed Work
  15. Ablation Study on Self-Attention Modules
  16. ...and 5 more sections

Figures (21)

  • Figure 1: Sample mid-day images along with the derived GCC (Green Chromatic Coordinate) and RCC (Red Chromatic Coordinate) values of NEON terrestrial sites within the “ NorthEast” domain. ROI (Region of Interest) describes the vegetation types: Deciduous Broadleaf (DB) and Evergreen Needleleaf (EN).
  • Figure 2: Examples of GAN-generated images available in the literature based on various datasets.
  • Figure 3: Outline of our proposed approach: Generator inputs a random noise vector, a random GCC value sampled within the range of GCC values for a specific ROI of the forest site under consideration, and the corresponding ROI image to generate a synthetic image, which satisfies the given GCC over the given ROI. Discriminator inputs the real or synthetic image and its corresponding GCC value and the ROI image to estimate the probability of the input image being real.
  • Figure 4: Our GAN architecture: The generator uses transposed convolution layers for upsampling, while the discrimator uses convolution layers for downsampling. Self-attention modules are incorporated into both the generator and discriminator to enhance the quality of the synthetic images (ablation study is conducted in Section \ref{['subsubsec:ablation']}). Spectral normalization, along with batch normalization, is applied to improve training stability.
  • Figure 5: The overall process employed by the generator: The generator model produces a synthetic image conditioned on an ROI image and a GCC value. The quality of the generated synthetic image is assessed by the SSIM score, which compares the synthetic image with the corresponding real image.
  • ...and 16 more figures