Robust soybean seed yield estimation using high-throughput ground robot videos

TL;DR

This work tackles the high cost and labor intensity of soybean yield data collection by leveraging ground-robot video data and deep learning for seed counting and yield estimation. It introduces P2PNet-Yield, a two-stage architecture that uses a P2PNet-Soy backbone for seed counting and a regression head to estimate plot yield, trained on a three-year, large-field dataset collected with fisheye cameras. Key innovations include fisheye distortion correction, data augmentation with camera sensor effects, and spatial adjustment to account for environmental variation, achieving strong genotype ranking performance and notable time/cost reductions. The approach demonstrates practical utility for plant breeding programs, enabling scalable, high-throughput yield estimation and ranking, with potential integration into broader phenotyping pipelines and field networks.

Abstract

We present a novel method for soybean (Glycine max (L.) Merr.) yield estimation leveraging high throughput seed counting via computer vision and deep learning techniques. Traditional methods for collecting yield data are labor-intensive, costly, prone to equipment failures at critical data collection times, and require transportation of equipment across field sites. Computer vision, the field of teaching computers to interpret visual data, allows us to extract detailed yield information directly from images. By treating it as a computer vision task, we report a more efficient alternative, employing a ground robot equipped with fisheye cameras to capture comprehensive videos of soybean plots from which images are extracted in a variety of development programs. These images are processed through the P2PNet-Yield model, a deep learning framework where we combined a Feature Extraction Module (the backbone of the P2PNet-Soy) and a Yield Regression Module to estimate seed yields of soybean plots. Our results are built on three years of yield testing plot data - 8500 in 2021, 2275 in 2022, and 650 in 2023. With these datasets, our approach incorporates several innovations to further improve the accuracy and generalizability of the seed counting and yield estimation architecture, such as the fisheye image correction and data augmentation with random sensor effects. The P2PNet-Yield model achieved a genotype ranking accuracy score of up to 83%. It demonstrates up to a 32% reduction in time to collect yield data as well as costs associated with traditional yield estimation, offering a scalable solution for breeding programs and agricultural productivity enhancement.

Figures (5)

• Figure 1: Example of our spatial adjustment grid pattern. Highlighted cells represent plots used in the spatial adjustment. Center cell that is not highlighted represents the cell being adjusted.
• Figure 2: This figure demonstrates the pipeline from in-field data collection and the post-processing needed for use in our P2PNet-Yield model. The first figure shows our Terrasentia robot operating in a mature soybean field. As the robot moves through the field, the two side-mounted cameras collect fisheye video data. Individual frames are then extracted, corrected for fisheye distortion, and cropped to remove blurry edges.
• Figure 4: Example of an expertly annotated image. All seeds clearly discernible to the naked eye were annotated using point annotations. This image represents a frame that has been calibrated for fisheye distortion and has been cropped.
• Figure 5: Our architecture, P2PNet-Yield, for soybean yield estimation. Training consists of two phases: first, training the P2PNet-Soy model so that its backbone (used as our Feature Extraction Module) can extract useful information related to soybean seeds in the foreground; second, training our Yield Regression Module to estimate yield values from the output feature maps of the Feature Extraction Module.
• Figure 8: Correlation between estimated TSC and estimated yield for the 650 plots from the 2023 F7 field. $R^2$ value of 0.06 and a correlation coefficient of 0.25.