Soybean Maturity Prediction using 2D Contour Plots from Drone based Time Series Imagery

TL;DR

The paper addresses the labor-intensive and subjective nature of soybean RM assessment in breeding programs by introducing a compact 2D hue-contour representation of time-series UAV RGB imagery as input to CNNs. It leverages SMOTE balancing and augmentation, training from scratch with ResNet34 to predict RM across 4, 5, and 7 class schemes, achieving up to 85% accuracy and near-perfect top-2 performance, with hierarchical and multi-temporal strategies enhancing fine-grained 7-class results. The study analyzes greenness dynamics via the ExG index (ExG = 2G − R − B) and shows associations between greenness loss rate, RM, and yield, while demonstrating that as few as three drone flights can suffice for accurate predictions, promising substantial operational savings in breeding programs. Overall, the work offers a scalable, objective framework for RM classification that integrates time-series greenness signals with deep learning, with practical implications for resource-efficient phenomics pipelines.

Abstract

Plant breeding programs require assessments of days to maturity for accurate selection and placement of entries in appropriate tests. In the early stages of the breeding pipeline, soybean breeding programs assign relative maturity ratings to experimental varieties that indicate their suitable maturity zones. Traditionally, the estimation of maturity value for breeding varieties has involved breeders manually inspecting fields and assessing maturity value visually. This approach relies heavily on rater judgment, making it subjective and time-consuming. This study aimed to develop a machine-learning model for evaluating soybean maturity using UAV-based time-series imagery. Images were captured at three-day intervals, beginning as the earliest varieties started maturing and continuing until the last varieties fully matured. The data collected for this experiment consisted of 22,043 plots collected across three years (2021 to 2023) and represent relative maturity groups 1.6 - 3.9. We utilized contour plot images extracted from the time-series UAV RGB imagery as input for a neural network model. This contour plot approach encoded the temporal and spatial variation within each plot into a single image. A deep learning model was trained to utilize this contour plot to predict maturity ratings. This model significantly improves accuracy and robustness, achieving up to 85% accuracy. We also evaluate the model's accuracy as we reduce the number of time points, quantifying the trade-off between temporal resolution and maturity prediction. The predictive model offers a scalable, objective, and efficient means of assessing crop maturity, enabling phenomics and ML approaches to reduce the reliance on manual inspection and subjective assessment. This approach enables the automatic prediction of relative maturity ratings in a breeding program, saving time and resources.

Figures (11)

• Figure 1: Maximum and minimum daily temperatures for 2021 (a), 2022 (b), and 2023 (c) during the period of time UAS flights were being conducted. Dashed lines represent UAS flight time points (TP).
• Figure 2: Example of a random plot before (a) and after (b) subsetting. Points between and including the TP max and TP min are subsetted, and a linear regression is calculated for these points. The slope of this regression line is then calculated to represent the linear rate of greenness loss over time.
• Figure 3: Overview of the workflow of 2D contour plot generation for the ML model. First, hue values and pixel counts are calculated for each time point. This workflow has feature extraction of the image as hue value, creating a 2D contour plot. For the final 2D contour plot, the hue range was cropped to 0-100.
• Figure 4: (a) Original Dataset Before SMOTE: This graph illustrates the class imbalance present in the original dataset (for the 5 class prediction scenario). The y-axis represents the number of samples in each class, while the x-axis labels each class from 1 to 5. Class 1 has the fewest samples (664), while Class 4 has the most (4893). (b) Resampled Dataset After SMOTE: This graph shows the dataset after applying SMOTE to balance the number of samples across all classes. Each class now has an equal number of samples (4893), as indicated on the y-axis, with class labels shown on the x-axis from 1 to 5.
• Figure 5: (a) This figure illustrates an example of how SMOTE works with two classes. It shows the distribution of the majority class (green), minority class (blue), and the synthetic samples generated by SMOTE (red) to balance the class distribution. (b) The left image is the original contour plot image. On the right side, there is the augmented image, where random masking and color-jitter are applied with the parameters: brightness = 0, contrast = 0.1, saturation = 0.2, and hue = 0.1. This augmentation introduces subtle variations in color and masks certain regions of the image, enhancing dataset diversity.