Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

3D Reconstruction-Based Seed Counting of Sorghum Panicles for Agricultural Inspection

Harry Freeman, Eric Schneider, Chung Hee Kim, Moonyoung Lee, George Kantor

TL;DR

This work tackles non-destructive sorghum seed counting by constructing high-quality 3D panicle models that leverage seeds as semantic landmarks in 2D and 3D. It introduces a seed-centered reconstruction pipeline, a cloud-only reconstruction quality metric, and a 3D counting method that uses DBSCAN clustering, Gaussian density smoothing, and local maxima to identify seed centers. Experimental results on a dataset of 100 panicles show a strong linear relationship between 3D counts and ground-truth seeds (R² = 0.875) with a RMSE of 295 seeds, and a seed-weight correlation of R² = 0.819 (RMSE ≈ 8.5 g). The study demonstrates that 3D counting outperforms 2D extrapolation, justifying the added hardware complexity for improved accuracy in yield-related phenotyping and robotic agriculture applications.

Abstract

In this paper, we present a method for creating high-quality 3D models of sorghum panicles for phenotyping in breeding experiments. This is achieved with a novel reconstruction approach that uses seeds as semantic landmarks in both 2D and 3D. To evaluate the performance, we develop a new metric for assessing the quality of reconstructed point clouds without having a ground-truth point cloud. Finally, a counting method is presented where the density of seed centers in the 3D model allows 2D counts from multiple views to be effectively combined into a whole-panicle count. We demonstrate that using this method to estimate seed count and weight for sorghum outperforms count extrapolation from 2D images, an approach used in most state of the art methods for seeds and grains of comparable size.

3D Reconstruction-Based Seed Counting of Sorghum Panicles for Agricultural Inspection

TL;DR

This work tackles non-destructive sorghum seed counting by constructing high-quality 3D panicle models that leverage seeds as semantic landmarks in 2D and 3D. It introduces a seed-centered reconstruction pipeline, a cloud-only reconstruction quality metric, and a 3D counting method that uses DBSCAN clustering, Gaussian density smoothing, and local maxima to identify seed centers. Experimental results on a dataset of 100 panicles show a strong linear relationship between 3D counts and ground-truth seeds (R² = 0.875) with a RMSE of 295 seeds, and a seed-weight correlation of R² = 0.819 (RMSE ≈ 8.5 g). The study demonstrates that 3D counting outperforms 2D extrapolation, justifying the added hardware complexity for improved accuracy in yield-related phenotyping and robotic agriculture applications.

Abstract

In this paper, we present a method for creating high-quality 3D models of sorghum panicles for phenotyping in breeding experiments. This is achieved with a novel reconstruction approach that uses seeds as semantic landmarks in both 2D and 3D. To evaluate the performance, we develop a new metric for assessing the quality of reconstructed point clouds without having a ground-truth point cloud. Finally, a counting method is presented where the density of seed centers in the 3D model allows 2D counts from multiple views to be effectively combined into a whole-panicle count. We demonstrate that using this method to estimate seed count and weight for sorghum outperforms count extrapolation from 2D images, an approach used in most state of the art methods for seeds and grains of comparable size.
Paper Structure (14 sections, 1 equation, 14 figures, 1 algorithm)

This paper contains 14 sections, 1 equation, 14 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Overview
  5. Instance Segmentation
  6. Global Registration
  7. Counting
  8. 3D Reconstruction Metric
  9. Experimental Results
  10. Dataset
  11. 3D Reconstruction Quality
  12. Prediction of Sorghum Characteristics
  13. Benefits of 3D Data over 2D
  14. Conclusion

Figures (14)

  • Figure 1: 3D Reconstruction pipeline for the sorghum stalk
  • Figure 2: Example reconstruction results. (a) one of the original RGB images, (b) the colorized point cloud, (c) zoomed view of the colorized point cloud at the stem, mid-body, and tip. Some points of interest include the "8" on the stem label, and the body outline which matches the RGB outline well.
  • Figure 3: Matching mask structure with maximum IOU. Seed masks 1, seed masks 2, and their intersection are colored blue, yellow, and green in respective order.
  • Figure 4: (a) An example of a final point cloud seed mask, (b) zoomed seeds, (c) seed centers, (d) seed centers clustered with DBSCAN, and (e) final seed sites.
  • Figure 5: (a) Seed point cloud that has been put in a single cluster by DBSCAN, (b) seed centers from individual images, (c) seed points weighted by seed-center density, and (d) local maxima (pink) that have been chosen as seeds.
  • ...and 9 more figures