Modeling Time-Lapse Trajectories to Characterize Cranberry Growth

TL;DR

The study tackles scalable change monitoring in cranberry crops by modeling time-lapse growth with Time-Lapse Trajectory (TLT), a self-supervised framework that fine-tunes vision-transformer features into an interpretable $2$-D latent space via UMAP and learns temporal tracks from fixed spatial regions. It jointly learns time, variety, and treatment (fungicide; rot for berries) through pretext tasks, enables trajectory-based growth prediction, and introduces the Time-lapse Cranberry Dataset (TLC) with 8 varieties over 52 dates and associated agronomic annotations. Key contributions include the TLT framework, a latent-space trajectory modeling approach using a Bayesian Gaussian mixture, and a publicly available TLC dataset to support cranberry change-monitoring research. The method demonstrates robust time prediction, transferable generalization to unseen varieties, and explainability via Grad-CAM, offering practical tools for breeders and growers and potential applicability to other crops.

Abstract

Change monitoring is an essential task for cranberry farming as it provides both breeders and growers with the ability to analyze growth, predict yield, and make treatment decisions. However, this task is often done manually, requiring significant time on the part of a cranberry grower or breeder. Deep learning based change monitoring holds promise, despite the caveat of hard-to-interpret high dimensional features and hand-annotations for fine-tuning. To address this gap, we introduce a method for modeling crop growth based on fine-tuning vision transformers (ViTs) using a self-supervised approach that avoids tedious image annotations. We use a two-fold pretext task (time regression and class prediction) to learn a latent space for the time-lapse evolution of plant and fruit appearance. The resulting 2D temporal tracks provide an interpretable time-series model of crop growth that can be used to: 1) predict growth over time and 2) distinguish temporal differences of cranberry varieties. We also provide a novel time-lapse dataset of cranberry fruit featuring eight distinct varieties, observed 52 times over the growing season (span of around four months), annotated with information about fungicide application, yield, and rot. Our approach is general and can be applied to other crops and applications (code and dataset can be found at https://github. com/ronan-39/tlt/).

Figures (10)

• Figure 1: In our proposed time-lapse trajectories (TLT) method, image features are projected to an interpretable lower dimensional space that is organized by meaningful differences in crop attributes. This projection is learned by training for several pretext tasks.
• Figure 2: Overview of our time-lapse-trajectory method. During training, a frozen pre-trained feature extractor backbone is appended with an encoder, which is jointly trained with several prediction heads for pretext tasks. This encoder is used in conjunction with UMAP to project images into a space that preserves relationships between time, class, etc., based on selected pretext tasks. The temporal tracks for patches are plotted as dots in latent space, while the temporal tracks for berries are plotted with segmented berries shown in the berry-based latent space. Reducing features to 2 dimensions provides interpretability, enabling growers to make informed decisions about breeding and crop management.
• Figure 3: Example region from cranberry bog delineated with a PVC frame fiducial marker for repeatable imaging (filtered out in pre-processing).
• Figure 4: Example images from the TLC (time-lapse cranberry) dataset comprised of 16 delineated regions imaged on 52 dates spanning 108 days with 8 cranberry varieties, each with and without fungicide.
• Figure 5: Examples of harvested berries from the 8 different varieties in our dataset. Each variety is color coded in this paper.