WeedVision: Multi-Stage Growth and Classification of Weeds using DETR and RetinaNet for Precision Agriculture

TL;DR

This work tackles precise weed management by detecting and classifying 16 weed species across 11 growth weeks using DETR and RetinaNet. It introduces a large greenhouse dataset of 203,567 labeled images and conducts a head-to-head comparison, finding RetinaNet superior in both accuracy (mAP) and speed, enabling real-time, species-specific weed control. The results show accuracy improves with plant maturation, supporting practical deployment in precision agriculture. The study offers deployment guidance and outlines future directions, including transformer-based improvements and field-data expansion to broaden applicability and impact on sustainable farming.

Abstract

Weed management remains a critical challenge in agriculture, where weeds compete with crops for essential resources, leading to significant yield losses. Accurate detection of weeds at various growth stages is crucial for effective management yet challenging for farmers, as it requires identifying different species at multiple growth phases. This research addresses these challenges by utilizing advanced object detection models, specifically, the Detection Transformer (DETR) with a ResNet50 backbone and RetinaNet with a ResNeXt101 backbone, to identify and classify 16 weed species of economic concern across 174 classes, spanning their 11 weeks growth stages from seedling to maturity. A robust dataset comprising 203,567 images was developed, meticulously labeled by species and growth stage. The models were rigorously trained and evaluated, with RetinaNet demonstrating superior performance, achieving a mean Average Precision (mAP) of 0.907 on the training set and 0.904 on the test set, compared to DETR's mAP of 0.854 and 0.840, respectively. RetinaNet also outperformed DETR in recall and inference speed of 7.28 FPS, making it more suitable for real time applications. Both models showed improved accuracy as plants matured. This research provides crucial insights for developing precise, sustainable, and automated weed management strategies, paving the way for real time species specific detection systems and advancing AI-assisted agriculture through continued innovation in model development and early detection accuracy.

Figures (6)

• Figure 1: Soil preparation and labeling for planting weed seeds in pots inside the greenhouse. (a) shows the prepared pots with soil and pot stakes, (b) displays the close-up of the soil mix used for planting.
• Figure 2: Greenhouse environment with lighting, temperature, and watering setup.
• Figure 3: Growth stages example of four weed species. (a,b) ABUTH in week 1 and 11; (c,d) ERICA in week 1 and 11; (e,f) SETFA in week 1 and 11; (g,h) CYPES in week 1 and 11. Images show progression from seedling emergence to mature plants across different species.
• Figure 4: Data Augmentation process with original image, masked image, and bounding box, respectively, for ERICA (a,b,c) and AMAPA (d,e,f).
• Figure 5: Illustration of the labeling process for weed detection. The original image (a) shows the weed plant, followed by the selected leaf area (b), highlighted in blue, and the final image (c) with a bounding box and label (AMBEL_week_8).