Semi-Supervised Weed Detection for Rapid Deployment and Enhanced Efficiency

TL;DR

This work tackles the labeling bottleneck in agricultural weed detection by introducing a semi-supervised approach that blends multi-scale feature representation with adaptive pseudo-label assignment. A three-part method, comprising a multi-scale representation within YOLOv5, an adaptive pseudo-label strategy, and a mean-teacher based Multi-Scale Detector with epoch-aware adaptation, enables effective learning from limited labeled data. Across COCO and five weed datasets, as well as domain adaptation tasks, the approach achieves state-of-the-art or near-state-of-the-art performance, with substantial gains when labels are scarce. The method promises reduced labeling burden and faster deployment for weed detection in real-world farming, potentially improving precision agriculture efficacy and sustainability.

Abstract

Weeds present a significant challenge in agriculture, causing yield loss and requiring expensive control measures. Automatic weed detection using computer vision and deep learning offers a promising solution. However, conventional deep learning methods often require large amounts of labelled training data, which can be costly and time-consuming to acquire. This paper introduces a novel method for semi-supervised weed detection, comprising two main components. Firstly, a multi-scale feature representation technique is employed to capture distinctive weed features across different scales. Secondly, we propose an adaptive pseudo-label assignment strategy, leveraging a small set of labelled images during training. This strategy dynamically assigns confidence scores to pseudo-labels generated from unlabeled data. Additionally, our approach integrates epoch-corresponding and mixed pseudo-labels to further enhance the learning process. Experimental results on the COCO dataset and five prominent weed datasets -- CottonWeedDet12, CropAndWeed, Palmer amaranth, RadishWheat, and RoboWeedMap -- illustrate that our method achieves state-of-the-art performance in weed detection, even with significantly less labelled data compared to existing techniques. This approach holds the potential to alleviate the labelling burden and enhance the feasibility and deployment speed of deep learning for weed detection in real-world agricultural scenarios.

Figures (4)

• Figure 1: Performance of our proposed semi-supervised model compared to a supervised model, when trained on 1% of available labelled training data across COCO and several weed datasets. See \ref{['tab_1']} and \ref{['tab_2']} for more details.
• Figure 2: Multi-scale Detector
• Figure 3: Multi-scale feature extraction and fusion in the YOLOv5 detector. The input image is first interpolated to different scales (e.g., $640^2$ and $1280^2$) and split into several sub-images of the same size as the default input size ($1280^2 \rightarrow 4 \times 640^2$). For each scale, all sub-images are fed into the same backbone model, and the outputs (e.g., $4 \times f_x^2$) are merged into a feature map of the whole image ($2 \times f_x^2$). Feature maps of different scales are max-pooled to the original spatial size ($f_x^2$) and concatenated together. The final multi-scale feature has the same spatial shape as the single-scale feature while having a higher channel dimension (e.g., 1024 vs. 512). This process is repeated for the feature maps from the backbone (P5/32, P4/16, and P3/8), and then fuses these features through the feature fusion network (neck) to finally generate three feature maps PP3, PP4, and PP5. After the three feature maps PP3, PP4, and PP5 were sent to the prediction head (head), the confidence calculation and bounding-box regression were executed for each pixel in the feature map using the preset prior anchor, so as to obtain a multi-dimensional array (BBoxes) including object class, class confidence, box coordinates, width, and height information.
• Figure 4: The Adaptive Pseudo Label Assignment process in Semi-Supervised Object Detection (SSOD). The Multi-Scale Detector analyzes data and assigns pseudo labels, which are categorized as reliable or unreliable based on two thresholds. Unreliable labels are handled specifically to improve the model's performance. The overall loss function combines supervised and unsupervised loss, with a weighting factor to balance their contributions. Indicator functions are used to selectively apply loss components based on the pseudo label score. This process effectively leverages both labeled and unlabeled data, improving model performance by handling the inherent uncertainty in pseudo labels.