Deep learning meets tree phenology modeling: PhenoFormer vs. process-based models

TL;DR

The results demonstrate that deep learning has the potential to be a valuable methodological tool for accurate climate-phenology prediction, and PhenoFormer is a first promising step in improving phenological predictions before a complete understanding of the underlying physiological mechanisms is available.

Abstract

Phenology, the timing of cyclical plant life events such as leaf emergence and coloration, is crucial in the bio-climatic system. Climate change drives shifts in these phenological events, impacting ecosystems and the climate itself. Accurate phenology models are essential to predict the occurrence of these phases under changing climatic conditions. Existing methods include hypothesis-driven process models and data-driven statistical approaches. Process models account for dormancy stages and various phenology drivers, while statistical models typically rely on linear or traditional machine learning techniques. Research shows that process models often outperform statistical methods when predicting under climate conditions outside historical ranges, especially with climate change scenarios. However, deep learning approaches remain underexplored in climate phenology modeling. We introduce PhenoFormer, a neural architecture better suited than traditional statistical methods at predicting phenology under shift in climate data distribution, while also bringing significant improvements or performing on par to the best performing process-based models. Our numerical experiments on a 70-year dataset of 70,000 phenological observations from 9 woody species in Switzerland show that PhenoFormer outperforms traditional machine learning methods by an average of 13% R2 and 1.1 days RMSE for spring phenology, and 11% R2 and 0.7 days RMSE for autumn phenology, while matching or exceeding the best process-based models. Our results demonstrate that deep learning has the potential to be a valuable methodological tool for accurate climate-phenology prediction, and our PhenoFormer is a first promising step in improving phenological predictions before a complete understanding of the underlying physiological mechanisms is available.

Figures (10)

• Figure 1: Images of the nine species of our dataset. The images are taken during spring months after the unfolding of new leaves or emergence of needles. (Images taken from iNaturalist under CC-0 license.)
• Figure 2: Location of phenology observation sites of the Swiss Phenology Network, coloured by elevation.
• Figure 3: Number of phenological observations per year and per species in the Swiss Phenology Network archive for spring (a) and autumn (b). Note the addition of new species in year 1996, entailing a limitation in the available history for some species.
• Figure 4: Distribution of the average annual temperature ($^\circ C$) in the training (green), validation (orange), and test (red) sets for the four different dataset splits we investigate. Note the significant shift in climatic conditions for the structured temporal split (d).
• Figure 5: PhenoFormer. The input climate time series $\mathbf{X}_n$ is first embedded into a higher dimensional space with a shared linear layer that produces a sequence of vectors $\mathbf{E}_n$. We append a learnt token $V$ to the sequence, and process it with a transformer layer. We use the output of the transformer for the learnt token as global representation of the climate time series. The global representation is decoded into the predicted date with a linear layer. The MSE loss computed between the predicted dates and the SPN observation is then used to update the trainable parameters of PhenoFormer with gradient descent.