Improved Crop and Weed Detection with Diverse Data Ensemble Learning

TL;DR

This work proposes a novel ensemble framework using homogeneous stacking of base models and a trainable meta-architecture to combine their outputs, and identifies the UNET meta-architecture as the most effective in this context.

Abstract

Modern agriculture heavily relies on Site-Specific Farm Management practices, necessitating accurate detection, localization, and quantification of crops and weeds in the field, which can be achieved using deep learning techniques. In this regard, crop and weed-specific binary segmentation models have shown promise. However, uncontrolled field conditions limit their performance from one field to the other. To improve semantic model generalization, existing methods augment and synthesize agricultural data to account for uncontrolled field conditions. However, given highly varied field conditions, these methods have limitations. To overcome the challenges of model deterioration in such conditions, we propose utilizing data specific to other crops and weeds for our specific target problem. To achieve this, we propose a novel ensemble framework. Our approach involves utilizing different crop and weed models trained on diverse datasets and employing a teacher-student configuration. By using homogeneous stacking of base models and a trainable meta-architecture to combine their outputs, we achieve significant improvements for Canola crops and Kochia weeds on unseen test data, surpassing the performance of single semantic segmentation models. We identify the UNET meta-architecture as the most effective in this context. Finally, through ablation studies, we demonstrate and validate the effectiveness of our proposed model. We observe that including base models trained on other target crops and weeds can help generalize the model to capture varied field conditions. Lastly, we propose two novel datasets with varied conditions for comparisons.

Figures (6)

• Figure 1: (a) Sample images containing early and mid-stage Canola plants, (b) Canola pixels classified by traditional encode-decoder scheme: ResNet50-SegNet. It can be observed that some Canola plant pixels are misclassified as background class. (c) Our proposed framework addresses the false negatives and rightly classifies the majority of Canola pixels. (Best viewed on screen and when zoomed-in).
• Figure 2: Our proposed framework for the ensemble of the teachers-student model. $\beta$'s are the base models, which are fused using meta-architecture. Table \ref{['CH3:teacher_models']} presents the details of base models and respective datasets used for training. The student model is trained for a specific problem (such as Canola) to learn from the ensemble base models trained on different datasets.
• Figure 3: The real world challenging field conditions: the data insight and the challenges posed by uncontrolled field conditions like blurring, variable orientations of crop rows, changing ambient lighting conditions, equipment shadows and images collected during night time under auxiliary lights.
• Figure 4: The visual comparisons from $\beta_{W_{k}}$ and MUNE models. a) The Groundtruth images, b) The prediction of the $\beta_{W_{k}}$ while c) predictions of MUNE on images. The $\beta_{W_{k}}$ model detects early-stage Canola and narrow leaf as Kochia (false positives), whereas the ensemble model addresses this problem and removes false positives.
• Figure 5: (a) Ground truth, (b) The individual models: $\beta_{C_{c}^{E}}$ & $\beta_{C_{c}^{L}}$ misses some Canola plants in both early and late stages of the crop (c) our proposed MUNE framework detects missing Canola plants in highly varied field conditions.