Deep Learning-Based Direct Leaf Area Estimation using Two RGBD Datasets for Model Development

TL;DR

The paper addresses direct leaf area estimation from RGBD images captured with a mobile camera in real-world crop settings, focusing on quantifying leaf area as a phenotypic trait. It explores a deep learning approach that starts with image-processing-based area estimation on manually segmented leaves, then adapts Mask R-CNN to RGBD inputs to predict leaf area, using two datasets: attached leaves with top-angle views and detached single leaves. A two-backbone network is proposed to separate segmentation and area estimation tasks, and a 5-fold cross-validation is performed with unseen detached and attached leaf data, achieving an F1 score of 1.0 (IoA 90%) for detached-leaf segmentation and an R^2 of 0.81 for area estimation; for unseen plant data, F1 is 0.59 and R^2 is 0.57. The findings indicate that leveraging attached leaves with ground-truth area improves results, demonstrating the practical viability of deep learning-based direct leaf area estimation from RGBD in realistic settings.

Abstract

Estimation of a single leaf area can be a measure of crop growth and a phenotypic trait to breed new varieties. It has also been used to measure leaf area index and total leaf area. Some studies have used hand-held cameras, image processing 3D reconstruction and unsupervised learning-based methods to estimate the leaf area in plant images. Deep learning works well for object detection and segmentation tasks; however, direct area estimation of objects has not been explored. This work investigates deep learning-based leaf area estimation, for RGBD images taken using a mobile camera setup in real-world scenarios. A dataset for attached leaves captured with a top angle view and a dataset for detached single leaves were collected for model development and testing. First, image processing-based area estimation was tested on manually segmented leaves. Then a Mask R-CNN-based model was investigated, and modified to accept RGBD images and to estimate the leaf area. The detached-leaf data set was then mixed with the attached-leaf plant data set to estimate the single leaf area for plant images, and another network design with two backbones was proposed: one for segmentation and the other for area estimation. Instead of trying all possibilities or random values, an agile approach was used in hyperparameter tuning. The final model was cross-validated with 5-folds and tested with two unseen datasets: detached and attached leaves. The F1 score with 90% IoA for segmentation result on unseen detached-leaf data was 1.0, while R-squared of area estimation was 0.81. For unseen plant data segmentation, the F1 score with 90% IoA was 0.59, while the R-squared score was 0.57. The research suggests using attached leaves with ground truth area to improve the results.