PhenoAssistant: A Conversational Multi-Agent AI System for Automated Plant Phenotyping

TL;DR

PhenoAssistant tackles the barrier of complex, code-heavy plant phenotyping by deploying a multi-agent AI system that orchestrates image-based phenotype extraction, data visualization, and automated model training through natural language. It couples a manager LLM with a plant-focused toolkit, including a vision model zoo and automated training pipelines, supported by agents for analysis, plotting, reproducibility, and literature retrieval. The work demonstrates three case studies across Arabidopsis, potato, and wheat, and provides an evaluation indicating solid tool- and vision-model selection performance alongside reliable data analysis capabilities. Open-source availability and pipeline reproducibility features highlight the practical impact: democratizing AI-assisted plant phenotyping and enabling researchers to design and re-run analyses without deep computational expertise.

Abstract

Plant phenotyping increasingly relies on (semi-)automated image-based analysis workflows to improve its accuracy and scalability. However, many existing solutions remain overly complex, difficult to reimplement and maintain, and pose high barriers for users without substantial computational expertise. To address these challenges, we introduce PhenoAssistant: a pioneering AI-driven system that streamlines plant phenotyping via intuitive natural language interaction. PhenoAssistant leverages a large language model to orchestrate a curated toolkit supporting tasks including automated phenotype extraction, data visualisation and automated model training. We validate PhenoAssistant through several representative case studies and a set of evaluation tasks. By significantly lowering technical hurdles, PhenoAssistant underscores the promise of AI-driven methodologies to democratising AI adoption in plant biology.

Figures (7)

• Figure 1: Design of PhenoAssistant. Users provide data and task description to PhenoAssistant. The manager LLM creates a step-by-step plan, selects and executes appropriate tools, then summarises the tool outputs to fulfil the task. Users retain full control to refine intermediate steps as needed. The icon above "Manager" is adapted from surang from www.flaticon.com, and those beside "Toolkit" and "Outputs" are adapted from Freepik from www.flaticon.com.
• Figure 2: Case Study 1: A. thaliana growth pattern analysis. PhenoAssistant automatically completes five tasks: computing phenotypes from images, plotting phenotypic statistics, analysing a generated plot, performing statistical tests for different ecotypes, and comparing findings with literature. Each task is presented as task description (grey), tools used by PhenoAssistant (blue), and results (white).
• Figure 3: Case Study 2: Potato leaf area and dry weight correlation analysis. In response to the user's requests, PhenoAssistant first extracts phenotypes from the provided data and then compares correlations between different plant-related variables.
• Figure 4: Case Study 3: Automatic model training for nutrient deficiency identification. When no suitable model is available to solve a given task, PhenoAssistant first prompts the user to provide a dataset in the desired format. It then automatically applies data preprocessing, followed by training and evaluating the model. The trained model is saved in the model zoo for future use.
• Figure 5: Two examples of prompting ChatGPT to extract projected leaf area and leaf count from a top-view image of an A. thaliana.