Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative diffusion models for agricultural AI: plant image generation, indoor-to-outdoor translation, and expert preference alignment

Da Tan, Michael Beck, Christopher P. Bidinosti, Robert H. Gulden, Christopher J. Henry

TL;DR

The paper tackles data scarcity in agricultural AI by proposing a unified diffusion-based framework that combines text-conditioned plant image generation, indoor-to-outdoor translation, and expert-preference aligned fine-tuning. It demonstrates that synthetic and translated imagery can improve downstream tasks such as phenotype classification and weed detection, while highlighting trade-offs between objective metrics and subjective quality. The approach provides a practical pathway to data-efficient, domain-aligned agricultural AI, capable of bridging indoor lab data and real-field conditions with improved robustness and expert-guided realism. Overall, the work showcases how diffusion models can be adapted and guided for tangible gains in plant phenotyping and precision agriculture.

Abstract

The success of agricultural artificial intelligence depends heavily on large, diverse, and high-quality plant image datasets, yet collecting such data in real field conditions is costly, labor intensive, and seasonally constrained. This paper investigates diffusion-based generative modeling to address these challenges through plant image synthesis, indoor-to-outdoor translation, and expert preference aligned fine tuning. First, a Stable Diffusion model is fine tuned on captioned indoor and outdoor plant imagery to generate realistic, text conditioned images of canola and soybean. Evaluation using Inception Score, Frechet Inception Distance, and downstream phenotype classification shows that synthetic images effectively augment training data and improve accuracy. Second, we bridge the gap between high resolution indoor datasets and limited outdoor imagery using DreamBooth-based text inversion and image guided diffusion, generating translated images that enhance weed detection and classification with YOLOv8. Finally, a preference guided fine tuning framework trains a reward model on expert scores and applies reward weighted updates to produce more stable and expert aligned outputs. Together, these components demonstrate a practical pathway toward data efficient generative pipelines for agricultural AI.

Generative diffusion models for agricultural AI: plant image generation, indoor-to-outdoor translation, and expert preference alignment

TL;DR

The paper tackles data scarcity in agricultural AI by proposing a unified diffusion-based framework that combines text-conditioned plant image generation, indoor-to-outdoor translation, and expert-preference aligned fine-tuning. It demonstrates that synthetic and translated imagery can improve downstream tasks such as phenotype classification and weed detection, while highlighting trade-offs between objective metrics and subjective quality. The approach provides a practical pathway to data-efficient, domain-aligned agricultural AI, capable of bridging indoor lab data and real-field conditions with improved robustness and expert-guided realism. Overall, the work showcases how diffusion models can be adapted and guided for tangible gains in plant phenotyping and precision agriculture.

Abstract

The success of agricultural artificial intelligence depends heavily on large, diverse, and high-quality plant image datasets, yet collecting such data in real field conditions is costly, labor intensive, and seasonally constrained. This paper investigates diffusion-based generative modeling to address these challenges through plant image synthesis, indoor-to-outdoor translation, and expert preference aligned fine tuning. First, a Stable Diffusion model is fine tuned on captioned indoor and outdoor plant imagery to generate realistic, text conditioned images of canola and soybean. Evaluation using Inception Score, Frechet Inception Distance, and downstream phenotype classification shows that synthetic images effectively augment training data and improve accuracy. Second, we bridge the gap between high resolution indoor datasets and limited outdoor imagery using DreamBooth-based text inversion and image guided diffusion, generating translated images that enhance weed detection and classification with YOLOv8. Finally, a preference guided fine tuning framework trains a reward model on expert scores and applies reward weighted updates to produce more stable and expert aligned outputs. Together, these components demonstrate a practical pathway toward data efficient generative pipelines for agricultural AI.
Paper Structure (41 sections, 9 equations, 8 figures, 7 tables)

This paper contains 41 sections, 9 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Theoretical Preliminaries
  3. Variational Autoencoders
  4. Diffusion Models and Latent Diffusion Models
  5. DreamBooth
  6. Overview of Methodology
  7. Text-Conditioned Image Generation
  8. Datasets and Preprocessing
  9. Text-Conditioned Diffusion Model Fine-Tuning
  10. Evaluation Metrics and Baseline Comparison
  11. Image Generation Results
  12. Qualitative Results
  13. Quantitative Evaluation
  14. Downstream Machine Learning Evaluation
  15. Dataset and Experiment Setup
  16. ...and 26 more sections

Figures (8)

  • Figure 1: Overview of the proposed framework. (a) Diffusion-based generative workflow encompassing text-conditioned image generation, image translation, and preference-guided fine-tuning. (b) Data augmentation process integrating synthetic and translated images for downstream tasks. (c) Agricultural applications including crop disease phenotyping and weed detection.
  • Figure 2: Text-conditioned image generation results. (a) Comparison between generated and real images for both indoor and outdoor canola plants. (b) Inception Score (IS) and Fréchet Inception Distance (FID) comparison with three baselines (Imagen, GigaGAN, DALL·E) for indoor and outdoor datasets. (c) Examples of generated vs. real images for downstream machine learning datasets: tomato (PlantVillage) and maize (CropDiseases). (d) Classification accuracy across different synthetic-to-real ratios, showing consistent improvement with data augmentation.
  • Figure 3: Overview of the dataset construction and translation workflow for the downstream indoor-to-outdoor soybean image translation. The process consists of: (1) removing background of the indoor soybean images, (2) cropping high quality plant patches, (3) compositing multiple patches onto a background canvas to mimic field-like arrangements, and (4) generating realistic outdoor translations using an image-conditioned diffusion model.
  • Figure 4: Indoor-to-Outdoor Image Translation and Downstream Detection. (a) Examples of translated images under different text prompts simulating environmental variations (lighting, soil, developmental stage, arrangement). (b) YOLOv8n detection and classification examples showing accurate bounding boxes for soybean and weed plants. (c) Detection metrics (precision, recall, and mAP50) across three weed species and varying synthetic-to-real ratios. Translation-based augmentation consistently improves model performance.
  • Figure 5: Preference-Guided Fine-Tuning of the Diffusion Model. (a) Workflow of the two-stage reward alignment pipeline integrating expert scoring, reward modeling, and weighted supervised fine-tuning. (b) Quantitative results showing the scatter of predicted vs. true rewards. (c) The training curve of mean reward score across epochs in preference-guide model fine-tuning. (d) Visual comparison between base and preference-guided models for identical prompts.
  • ...and 3 more figures