Enhancing weed detection performance by means of GenAI-based image augmentation

TL;DR

This work addresses the data scarcity hindering robust real-time weed detection on edge devices by introducing GenAI-based image augmentation using a SAM-guided Stable Diffusion pipeline to generate diverse synthetic weed-crop scenes. The approach is evaluated on compact YOLO variants (YOLOv8n, YOLOv9t, YOLOv10-N) with and without COCO pretraining, showing improvements in $mAP_{50}$ and $mAP_{50-95}$ across models; notably, larger gains occur when training from scratch (e.g., $mAP_{50}$ gains of about 20–30 percentage points for some models). Traditional augmentations remain valuable, but synthetic data provides broader diversity and better generalization in heterogeneous field conditions, albeit with higher compute and quality-control requirements. The findings demonstrate the potential of synthetic data to reduce real-data annotation needs and enhance edge-ready weed detection, informing future hybrid augmentation strategies and efficiency optimizations for embedded deployment.

Abstract

Precise weed management is essential for sustaining crop productivity and ecological balance. Traditional herbicide applications face economic and environmental challenges, emphasizing the need for intelligent weed control systems powered by deep learning. These systems require vast amounts of high-quality training data. The reality of scarcity of well-annotated training data, however, is often addressed through generating more data using data augmentation. Nevertheless, conventional augmentation techniques such as random flipping, color changes, and blurring lack sufficient fidelity and diversity. This paper investigates a generative AI-based augmentation technique that uses the Stable Diffusion model to produce diverse synthetic images that improve the quantity and quality of training datasets for weed detection models. Moreover, this paper explores the impact of these synthetic images on the performance of real-time detection systems, thus focusing on compact CNN-based models such as YOLO nano for edge devices. The experimental results show substantial improvements in mean Average Precision (mAP50 and mAP50-95) scores for YOLO models trained with generative AI-augmented datasets, demonstrating the promising potential of synthetic data to enhance model robustness and accuracy.

Figures (3)

• Figure 1: Samples of pseudo-RGB images: (a) and (b) are from described real-world data sets, while (c) and (d) are synthetic images generated by the image generation pipeline. The dataset comprises main crop Sugar beet and weeds from both real-world and synthetic datasets.
• Figure 2: Examples of problematic synthetic images illustrating various issues: (a) and (b) irregular phenotypic details (e.g., abnormalities in leaf structure); (c) deformed and distorted features; and (d) absence of meaningful content or occurrence of nonsensical patterns.
• Figure 3: The weed detection pipeline consists of 4 steps: data preparation (a), data augmentation (b), model training (c), and evaluation (d). Step a) involves data collection via a camera on a smart sprayer and text-prompting with a fine-tuned Stable Diffusion model. Synthetic images are annotated by a fine-tuned YOLOv8x model using a real-world manually annotated dataset. Step b) applies two types of augmentation: i) Traditional techniques like copy-paste, mixup, changing HSV, flipping & rotation, and ii) augmentation with synthetic images, increasing from 10% to 200% in 10% increments. In step c), three compact YOLO models—YOLOv8n, YOLOv9t, and YOLOv10-N—are used. Finally, step d) evaluates the augmented images.