Agtech Framework for Cranberry-Ripening Analysis Using Vision Foundation Models

TL;DR

This work presents a time-series, vision-based framework to quantify cranberry ripening using both aerial drone imagery and ground-based photography. It combines weakly supervised segmentation, photometric calibration, Vision Transformer features, and a UMAP appearance manifold to produce interpretable ripening trajectories at bog and single-berry scales. The authors introduce the CRAID+ dataset and demonstrate variety-level ripening differences and per-berry trajectories, enabling high-throughput phenotyping and irrigation decision support to mitigate overheating. The approach is scalable, applicable to other crops, and publicly releases the gathered datasets to promote broader adoption in agricultural analytics.

Abstract

Agricultural domains are being transformed by recent advances in AI and computer vision that support quantitative visual evaluation. Using aerial and ground imaging over a time series, we develop a framework for characterizing the ripening process of cranberry crops, a crucial component for precision agriculture tasks such as comparing crop breeds (high-throughput phenotyping) and detecting disease. Using drone imaging, we capture images from 20 waypoints across multiple bogs, and using ground-based imaging (hand-held camera), we image same bog patch using fixed fiducial markers. Both imaging methods are repeated to gather a multi-week time series spanning the entire growing season. Aerial imaging provides multiple samples to compute a distribution of albedo values. Ground imaging enables tracking of individual berries for a detailed view of berry appearance changes. Using vision transformers (ViT) for feature detection after segmentation, we extract a high dimensional feature descriptor of berry appearance. Interpretability of appearance is critical for plant biologists and cranberry growers to support crop breeding decisions (e.g.\ comparison of berry varieties from breeding programs). For interpretability, we create a 2D manifold of cranberry appearance by using a UMAP dimensionality reduction on ViT features. This projection enables quantification of ripening paths and a useful metric of ripening rate. We demonstrate the comparison of four cranberry varieties based on our ripening assessments. This work is the first of its kind and has future impact for cranberries and for other crops including wine grapes, olives, blueberries, and maize. Aerial and ground datasets are made publicly available.

Figures (10)

• Figure 1: Cranberry bog at the measurement site. (Left) Cranberry harvesting. (Right) Drone at bog for in-field cranberry measurements during the growing season.
• Figure 2: An example segmentation of cranberry images using the Segment Anything Model SAMravi2024sam without point-click prompts (automatic mask generation). Notice that in addition to cranberries, surrounding leaves and other structures are segmented.
• Figure 3: Example of breeding plots (drone view) that are typically evaluated manually. Planting design permits approx 3500 plots/ha, and this entire block is approximately 2 ha. Convenient quantitative evaluation can be supported by our vision-based ripening assessment framework. Location removed for blind review.
• Figure 4: Drone images from multi-temporal drone-scout imaging. Weekly inspection of multiple cranberry bogs over the late July/September growing season for four varieties (Mullica Queen, Stevens, Crimson Queen, Haines). (Left to Right) Imaging Dates for 2022: 7/27, 8/2, 8/16, 8/25, 8/31, 9/9.
• Figure 5: Ground-based imaging (hand-held DSLR camera) of the same region in a cranberry bog was done over 27 sessions (almost daily during the growing season) to show the appearance of individual cranberries over time. The semi-permanent PVC frame enabled identification of the imaged region over time.