Leaf Angle Estimation using Mask R-CNN and LETR Vision Transformer
Venkat Margapuri, Prapti Thapaliya, Trevor Rife
TL;DR
This work addresses the challenge of tedious on-field leaf-angle measurement by proposing an automated pipeline that combines Mask R-CNN-based ROI extraction with LETR, a DETR-inspired vision transformer, to estimate leaf angles from field images. The approach is validated on 1,827 images from Summer 2015 Ames ULA/MLA captured with FieldBook, with leaf-angle estimates showing near-perfect agreement to manual measurements, as evidenced by cosine similarities around $0.98$–$0.99$ and small angular variances. The results demonstrate the feasibility of on-site LA estimation and point toward practical deployment in a mobile app, potentially enabling high-throughput phenotyping and accelerated breeding for upright leaf architectures. Future work includes adapting the ROI detector for mobile use (e.g., Mobilenet backbones) and integrating the method into FieldBook for real-time field phenotyping.
Abstract
Modern day studies show a high degree of correlation between high yielding crop varieties and plants with upright leaf angles. It is observed that plants with upright leaf angles intercept more light than those without upright leaf angles, leading to a higher rate of photosynthesis. Plant scientists and breeders benefit from tools that can directly measure plant parameters in the field i.e. on-site phenotyping. The estimation of leaf angles by manual means in a field setting is tedious and cumbersome. We mitigate the tedium using a combination of the Mask R-CNN instance segmentation neural network, and Line Segment Transformer (LETR), a vision transformer. The proposed Computer Vision (CV) pipeline is applied on two image datasets, Summer 2015-Ames ULA and Summer 2015- Ames MLA, with a combined total of 1,827 plant images collected in the field using FieldBook, an Android application aimed at on-site phenotyping. The leaf angles estimated by the proposed pipeline on the image datasets are compared to two independent manual measurements using ImageJ, a Java-based image processing program developed at the National Institutes of Health and the Laboratory for Optical and Computational Instrumentation. The results, when compared for similarity using the Cosine Similarity measure, exhibit 0.98 similarity scores on both independent measurements of Summer 2015-Ames ULA and Summer 2015-Ames MLA image datasets, demonstrating the feasibility of the proposed pipeline for on-site measurement of leaf angles.