Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-Throughput Phenotyping using Computer Vision and Machine Learning

Vivaan Singhvi, Langalibalele Lunga, Pragya Nidhi, Chris Keum, Varrun Prakash

TL;DR

High-throughput phenotyping aims to non-destructively quantify plant traits at scale. The authors develop an end-to-end pipeline combining PaddleOCR for label reading, Segment Anything Model-based leaf segmentation, and RandomForest-based morphology classification to predict drought vs control treatments, while also attempting to use EXIF data for leaf-size estimation and phenotype-environment correlations. They report OCR accuracy of 94.31% non-null, morphology accuracy 62.82%, and treatment prediction accuracy 60.08%, but EXIF-based sizing and correlations are limited by metadata gaps. The work demonstrates automated data extraction and phenotype analysis from images, highlighting both the potential and the need for richer metadata to enable robust genotype–environment insights.

Abstract

High-throughput phenotyping refers to the non-destructive and efficient evaluation of plant phenotypes. In recent years, it has been coupled with machine learning in order to improve the process of phenotyping plants by increasing efficiency in handling large datasets and developing methods for the extraction of specific traits. Previous studies have developed methods to advance these challenges through the application of deep neural networks in tandem with automated cameras; however, the datasets being studied often excluded physical labels. In this study, we used a dataset provided by Oak Ridge National Laboratory with 1,672 images of Populus Trichocarpa with white labels displaying treatment (control or drought), block, row, position, and genotype. Optical character recognition (OCR) was used to read these labels on the plants, image segmentation techniques in conjunction with machine learning algorithms were used for morphological classifications, machine learning models were used to predict treatment based on those classifications, and analyzed encoded EXIF tags were used for the purpose of finding leaf size and correlations between phenotypes. We found that our OCR model had an accuracy of 94.31% for non-null text extractions, allowing for the information to be accurately placed in a spreadsheet. Our classification models identified leaf shape, color, and level of brown splotches with an average accuracy of 62.82%, and plant treatment with an accuracy of 60.08%. Finally, we identified a few crucial pieces of information absent from the EXIF tags that prevented the assessment of the leaf size. There was also missing information that prevented the assessment of correlations between phenotypes and conditions. However, future studies could improve upon this to allow for the assessment of these features.

High-Throughput Phenotyping using Computer Vision and Machine Learning

TL;DR

High-throughput phenotyping aims to non-destructively quantify plant traits at scale. The authors develop an end-to-end pipeline combining PaddleOCR for label reading, Segment Anything Model-based leaf segmentation, and RandomForest-based morphology classification to predict drought vs control treatments, while also attempting to use EXIF data for leaf-size estimation and phenotype-environment correlations. They report OCR accuracy of 94.31% non-null, morphology accuracy 62.82%, and treatment prediction accuracy 60.08%, but EXIF-based sizing and correlations are limited by metadata gaps. The work demonstrates automated data extraction and phenotype analysis from images, highlighting both the potential and the need for richer metadata to enable robust genotype–environment insights.

Abstract

High-throughput phenotyping refers to the non-destructive and efficient evaluation of plant phenotypes. In recent years, it has been coupled with machine learning in order to improve the process of phenotyping plants by increasing efficiency in handling large datasets and developing methods for the extraction of specific traits. Previous studies have developed methods to advance these challenges through the application of deep neural networks in tandem with automated cameras; however, the datasets being studied often excluded physical labels. In this study, we used a dataset provided by Oak Ridge National Laboratory with 1,672 images of Populus Trichocarpa with white labels displaying treatment (control or drought), block, row, position, and genotype. Optical character recognition (OCR) was used to read these labels on the plants, image segmentation techniques in conjunction with machine learning algorithms were used for morphological classifications, machine learning models were used to predict treatment based on those classifications, and analyzed encoded EXIF tags were used for the purpose of finding leaf size and correlations between phenotypes. We found that our OCR model had an accuracy of 94.31% for non-null text extractions, allowing for the information to be accurately placed in a spreadsheet. Our classification models identified leaf shape, color, and level of brown splotches with an average accuracy of 62.82%, and plant treatment with an accuracy of 60.08%. Finally, we identified a few crucial pieces of information absent from the EXIF tags that prevented the assessment of the leaf size. There was also missing information that prevented the assessment of correlations between phenotypes and conditions. However, future studies could improve upon this to allow for the assessment of these features.
Paper Structure (26 sections, 1 equation, 17 figures, 3 tables)

This paper contains 26 sections, 1 equation, 17 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background Information
  3. Research Objective
  4. Related Works
  5. Advancements in High Throughput Phenotyping Techniques
  6. Machine Learning Approaches for Phenotypic Analysis
  7. Applications of High Throughput Phenotyping with Machine Learning
  8. Methodology
  9. Reading Labels with Optical Character Recognition
  10. Classifying Leaf Morphologies with Image Processing
  11. Treatment Predictions using Morphological Classifications
  12. Using EXIF Data to Assess Leaf Size and Analyze Correlations Between Morphologies and Environments
  13. Results and Discussion
  14. Reading Labels with Optical Character Recognition
  15. Classifying Leaf Morphologies with Image Processing
  16. ...and 11 more sections

Figures (17)

  • Figure 1: Different versions of images tried for optical character recognition, in order
  • Figure 2: Pipeline of image transformations for leaf approximation, read left to right. The last image visualizes detected contours
  • Figure 3: From leaf approximations with image processing to filtered leaf segmentations, from the SAM/ONNX Runtime (green leaves fit for classification)
  • Figure 4: Leaf shape reference from the article by nakano2020plant
  • Figure 5: Geographical plot of all plants using encoded EXIF GPS data.
  • ...and 12 more figures